Mercurial > public > mercurial-scm > hg

annotate contrib/automation/hgautomation/cli.py @ 51696:7f0cb9ee0534

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Backout accidental publication of a large range of revisions I accidentally published 25e7f9dcad0f::bd1483fd7088, this is the inverse.
author Rapha?l Gom?s <rgomes@octobus.net>
date Tue, 23 Jul 2024 10:02:46 +0200
parents 493034cc3265
children ca7bde5dbafb
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # cli.py - Command line interface for automation
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 import argparse
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
11 import concurrent.futures as futures
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 import os
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 import pathlib
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
14 import time
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 from . import (
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 aws,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 HGAutomation,
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
19 linux,
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
20 try_server,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 windows,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
25 SOURCE_ROOT = pathlib.Path(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
26 os.path.abspath(__file__)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
27 ).parent.parent.parent.parent
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28 DIST_PATH = SOURCE_ROOT / 'dist'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
31 def bootstrap_linux_dev(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
32 hga: HGAutomation, aws_region, distros=None, parallel=False
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
33 ):
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
34 c = hga.aws_connection(aws_region)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
35
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
36 if distros:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
37 distros = distros.split(',')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
38 else:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
39 distros = sorted(linux.DISTROS)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
40
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
41 # TODO There is a wonky interaction involving KeyboardInterrupt whereby
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
42 # the context manager that is supposed to terminate the temporary EC2
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
43 # instance doesn't run. Until we fix this, make parallel building opt-in
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
44 # so we don't orphan instances.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
45 if parallel:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
46 fs = []
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
47
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
48 with futures.ThreadPoolExecutor(len(distros)) as e:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
49 for distro in distros:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
50 fs.append(e.submit(aws.ensure_linux_dev_ami, c, distro=distro))
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
51
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
52 for f in fs:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
53 f.result()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
54 else:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
55 for distro in distros:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
56 aws.ensure_linux_dev_ami(c, distro=distro)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
57
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
58
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
59 def bootstrap_windows_dev(hga: HGAutomation, aws_region, base_image_name):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
61 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62 print('Windows development AMI available as %s' % image.id)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
65 def build_inno(
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
66 hga: HGAutomation,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
67 aws_region,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
68 arch,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
69 revision,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
70 version,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
71 base_image_name,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
72 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
74 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
80 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
81
48847
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
82 for a in arch:
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
83 windows.build_inno_installer(
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
84 instance.winrm_client,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
85 a,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
86 DIST_PATH,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
87 version=version,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
88 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
91 def build_wix(
44772
5e788dc7fb5d automation: support building Python 3 MSI installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44771
diff changeset
92 hga: HGAutomation,
5e788dc7fb5d automation: support building Python 3 MSI installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44771
diff changeset
93 aws_region,
5e788dc7fb5d automation: support building Python 3 MSI installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44771
diff changeset
94 arch,
5e788dc7fb5d automation: support building Python 3 MSI installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44771
diff changeset
95 revision,
5e788dc7fb5d automation: support building Python 3 MSI installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44771
diff changeset
96 version,
5e788dc7fb5d automation: support building Python 3 MSI installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44771
diff changeset
97 base_image_name,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
98 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
100 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
101 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
102
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
103 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
104 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
105
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
106 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
107
48847
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
108 for a in arch:
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
109 windows.build_wix_installer(
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
110 instance.winrm_client,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
111 a,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
112 DIST_PATH,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
113 version=version,
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
114 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
115
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
116
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
117 def build_windows_wheel(
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
118 hga: HGAutomation,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
119 aws_region,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
120 python_version,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
121 arch,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
122 revision,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
123 base_image_name,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
124 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
125 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
126 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
127 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
128
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
129 with aws.temporary_windows_dev_instances(c, image, 't3.medium') as insts:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
130 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
131
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
132 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
133
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
134 for py_version in python_version:
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
135 for a in arch:
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
136 windows.build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
137 instance.winrm_client, py_version, a, DIST_PATH
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
138 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
139
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
140
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
141 def build_all_windows_packages(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
142 hga: HGAutomation, aws_region, revision, version, base_image_name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
143 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
144 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
145 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
146 DIST_PATH.mkdir(exist_ok=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
147
48747
ae28d37f5969 automation: use m6i instances
Gregory Szorc <gregory.szorc@gmail.com>
parents: 45942
diff changeset
148 with aws.temporary_windows_dev_instances(c, image, 'm6i.large') as insts:
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
149 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
150
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
151 winrm_client = instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
152
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
153 windows.synchronize_hg(SOURCE_ROOT, revision, instance)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
154
48846
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48845
diff changeset
155 for py_version in ("3.7", "3.8", "3.9", "3.10"):
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
156 for arch in ("x86", "x64"):
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
157 windows.purge_hg(winrm_client)
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
158 windows.build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
159 winrm_client,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
160 python_version=py_version,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
161 arch=arch,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
162 dest_path=DIST_PATH,
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
163 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
164
48847
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
165 for arch in ('x86', 'x64'):
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
166 windows.purge_hg(winrm_client)
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
167 windows.build_inno_installer(
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
168 winrm_client, arch, DIST_PATH, version=version
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
169 )
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
170 windows.build_wix_installer(
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
171 winrm_client, arch, DIST_PATH, version=version
4561ec90d3c1 automation: delete code related to Python 2.7 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48846
diff changeset
172 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
173
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
174
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
175 def terminate_ec2_instances(hga: HGAutomation, aws_region):
42277
dd6a9723ae2b automation: don't create resources when deleting things
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
176 c = hga.aws_connection(aws_region, ensure_ec2_state=False)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
177 aws.terminate_ec2_instances(c.ec2resource)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
178
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
179
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
180 def purge_ec2_resources(hga: HGAutomation, aws_region):
42277
dd6a9723ae2b automation: don't create resources when deleting things
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
181 c = hga.aws_connection(aws_region, ensure_ec2_state=False)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
182 aws.remove_resources(c)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
183
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
184
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
185 def run_tests_linux(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
186 hga: HGAutomation,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
187 aws_region,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
188 instance_type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
189 python_version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
190 test_flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
191 distro,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
192 filesystem,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
193 ):
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
194 c = hga.aws_connection(aws_region)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
195 image = aws.ensure_linux_dev_ami(c, distro=distro)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
196
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
197 t_start = time.time()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
198
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
199 ensure_extra_volume = filesystem not in ('default', 'tmpfs')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
200
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
201 with aws.temporary_linux_dev_instances(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
202 c, image, instance_type, ensure_extra_volume=ensure_extra_volume
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
203 ) as insts:
51696
7f0cb9ee0534 Backout accidental publication of a large range of revisions
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51686
diff changeset
204
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
205 instance = insts[0]
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
206
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
207 linux.prepare_exec_environment(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
208 instance.ssh_client, filesystem=filesystem
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
209 )
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
210 linux.synchronize_hg(SOURCE_ROOT, instance, '.')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
211 t_prepared = time.time()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
212 linux.run_tests(instance.ssh_client, python_version, test_flags)
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
213 t_done = time.time()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
214
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
215 t_setup = t_prepared - t_start
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
216 t_all = t_done - t_start
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
217
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
218 print(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
219 'total time: %.1fs; setup: %.1fs; tests: %.1fs; setup overhead: %.1f%%'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
220 % (t_all, t_setup, t_done - t_prepared, t_setup / t_all * 100.0)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
221 )
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
222
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
223
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
224 def run_tests_windows(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
225 hga: HGAutomation,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
226 aws_region,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
227 instance_type,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
228 python_version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
229 arch,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
230 test_flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
231 base_image_name,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
232 ):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
233 c = hga.aws_connection(aws_region)
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
234 image = aws.ensure_windows_dev_ami(c, base_image_name=base_image_name)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
235
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
236 with aws.temporary_windows_dev_instances(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
237 c, image, instance_type, disable_antivirus=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
238 ) as insts:
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
239 instance = insts[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
240
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
241 windows.synchronize_hg(SOURCE_ROOT, '.', instance)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
242 windows.run_tests(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
243 instance.winrm_client, python_version, arch, test_flags
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
244 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
245
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
246
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
247 def publish_windows_artifacts(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
248 hg: HGAutomation,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
249 aws_region,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
250 version: str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
251 pypi: bool,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
252 mercurial_scm_org: bool,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
253 ssh_username: str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
254 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
255 windows.publish_artifacts(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
256 DIST_PATH,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
257 version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
258 pypi=pypi,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
259 mercurial_scm_org=mercurial_scm_org,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
260 ssh_username=ssh_username,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
261 )
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
262
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
263
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
264 def run_try(hga: HGAutomation, aws_region: str, rev: str):
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
265 c = hga.aws_connection(aws_region, ensure_ec2_state=False)
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
266 try_server.trigger_try(c, rev=rev)
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
267
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
268
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
269 def get_parser():
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
270 parser = argparse.ArgumentParser()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
271
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
272 parser.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
273 '--state-path',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
274 default='~/.hgautomation',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
275 help='Path for local state files',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
276 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
277 parser.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
278 '--aws-region',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
279 help='AWS region to use',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
280 default='us-west-2',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
281 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
282
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
283 subparsers = parser.add_subparsers()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
284
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
285 sp = subparsers.add_parser(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
286 'bootstrap-linux-dev',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
287 help='Bootstrap Linux development environments',
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
288 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
289 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
290 '--distros',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
291 help='Comma delimited list of distros to bootstrap',
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
292 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
293 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
294 '--parallel',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
295 action='store_true',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
296 help='Generate AMIs in parallel (not CTRL-c safe)',
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
297 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
298 sp.set_defaults(func=bootstrap_linux_dev)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
299
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
300 sp = subparsers.add_parser(
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
301 'bootstrap-windows-dev',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
302 help='Bootstrap the Windows development environment',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
303 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
304 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
305 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
306 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
307 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
308 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
309 sp.set_defaults(func=bootstrap_windows_dev)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
310
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
311 sp = subparsers.add_parser(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
312 'build-all-windows-packages',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
313 help='Build all Windows packages',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
314 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
315 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
316 '--revision',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
317 help='Mercurial revision to build',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
318 default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
319 )
42283
d137a3d5ad41 automation: add --version argument to build-all-windows-packages
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42277
diff changeset
320 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
321 '--version',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
322 help='Mercurial version string to use',
42283
d137a3d5ad41 automation: add --version argument to build-all-windows-packages
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42277
diff changeset
323 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
324 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
325 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
326 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
327 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
328 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
329 sp.set_defaults(func=build_all_windows_packages)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
330
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
331 sp = subparsers.add_parser(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
332 'build-inno',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
333 help='Build Inno Setup installer(s)',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
334 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
335 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
336 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
337 help='Architecture to build for',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
338 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
339 nargs='*',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
340 default=['x64'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
341 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
342 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
343 '--revision',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
344 help='Mercurial revision to build',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
345 default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
346 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
347 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
348 '--version',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
349 help='Mercurial version string to use in installer',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
350 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
351 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
352 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
353 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
354 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
355 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
356 sp.set_defaults(func=build_inno)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
357
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
358 sp = subparsers.add_parser(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
359 'build-windows-wheel',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
360 help='Build Windows wheel(s)',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
361 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
362 sp.add_argument(
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
363 '--python-version',
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
364 help='Python version to build for',
48846
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48845
diff changeset
365 choices={'3.7', '3.8', '3.9', '3.10'},
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
366 nargs='*',
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
367 default=['3.8'],
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
368 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
369 sp.add_argument(
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
370 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
371 help='Architecture to build for',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
372 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
373 nargs='*',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
374 default=['x64'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
375 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
376 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
377 '--revision',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
378 help='Mercurial revision to build',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
379 default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
380 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
381 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
382 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
383 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
384 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
385 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
386 sp.set_defaults(func=build_windows_wheel)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
387
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
388 sp = subparsers.add_parser('build-wix', help='Build WiX installer(s)')
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
389 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
390 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
391 help='Architecture to build for',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
392 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
393 nargs='*',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
394 default=['x64'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
395 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
396 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
397 '--revision',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
398 help='Mercurial revision to build',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
399 default='.',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
400 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
401 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
402 '--version',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
403 help='Mercurial version string to use in installer',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
404 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
405 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
406 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
407 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
408 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
409 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
410 sp.set_defaults(func=build_wix)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
411
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
412 sp = subparsers.add_parser(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
413 'terminate-ec2-instances',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
414 help='Terminate all active EC2 instances managed by us',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
415 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
416 sp.set_defaults(func=terminate_ec2_instances)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
417
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
418 sp = subparsers.add_parser(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
419 'purge-ec2-resources',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
420 help='Purge all EC2 resources managed by us',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
421 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
422 sp.set_defaults(func=purge_ec2_resources)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
423
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
424 sp = subparsers.add_parser(
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
425 'run-tests-linux',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
426 help='Run tests on Linux',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
427 )
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
428 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
429 '--distro',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
430 help='Linux distribution to run tests on',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
431 choices=linux.DISTROS,
43018
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42907
diff changeset
432 default='debian10',
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
433 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
434 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
435 '--filesystem',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
436 help='Filesystem type to use',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
437 choices={'btrfs', 'default', 'ext3', 'ext4', 'jfs', 'tmpfs', 'xfs'},
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
438 default='default',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
439 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
440 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
441 '--instance-type',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
442 help='EC2 instance type to use',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
443 default='c5.9xlarge',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
444 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
445 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
446 '--python-version',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
447 help='Python version to use',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
448 choices={
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
449 'system3',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
450 '3.5',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
451 '3.6',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
452 '3.7',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
453 '3.8',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
454 'pypy',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
455 'pypy3.5',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
456 'pypy3.6',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
457 },
48844
d79f0ce95c47 automation: make system3 the default for run-tests-linux
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48794
diff changeset
458 default='system3',
42285
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
459 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
460 sp.add_argument(
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
461 'test_flags',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
462 help='Extra command line flags to pass to run-tests.py',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
463 nargs='*',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
464 )
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
465 sp.set_defaults(func=run_tests_linux)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
466
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42283
diff changeset
467 sp = subparsers.add_parser(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
468 'run-tests-windows',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
469 help='Run tests on Windows',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
470 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
471 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
472 '--instance-type',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
473 help='EC2 instance type to use',
48747
ae28d37f5969 automation: use m6i instances
Gregory Szorc <gregory.szorc@gmail.com>
parents: 45942
diff changeset
474 default='m6i.large',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
475 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
476 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
477 '--python-version',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
478 help='Python version to use',
48846
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48845
diff changeset
479 choices={'3.5', '3.6', '3.7', '3.8', '3.9', '3.10'},
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48845
diff changeset
480 default='3.9',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
481 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
482 sp.add_argument(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
483 '--arch',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
484 help='Architecture to test',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
485 choices={'x86', 'x64'},
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
486 default='x64',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
487 )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
488 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
489 '--test-flags',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
490 help='Extra command line flags to pass to run-tests.py',
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
491 )
42648
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
492 sp.add_argument(
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
493 '--base-image-name',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
494 help='AMI name of base image',
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
495 default=aws.WINDOWS_BASE_IMAGE_NAME,
d80edcb0b30c automation: make Windows base image name configurable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42285
diff changeset
496 )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
497 sp.set_defaults(func=run_tests_windows)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
498
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
499 sp = subparsers.add_parser(
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
500 'publish-windows-artifacts',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
501 help='Publish built Windows artifacts (wheels, installers, etc)',
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
502 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
503 sp.add_argument(
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
504 '--no-pypi',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
505 dest='pypi',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
506 action='store_false',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
507 default=True,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
508 help='Skip uploading to PyPI',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
509 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
510 sp.add_argument(
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
511 '--no-mercurial-scm-org',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
512 dest='mercurial_scm_org',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
513 action='store_false',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
514 default=True,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
515 help='Skip uploading to www.mercurial-scm.org',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
516 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
517 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
518 '--ssh-username',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
519 help='SSH username for mercurial-scm.org',
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
520 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
521 sp.add_argument(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
522 'version',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45693
diff changeset
523 help='Mercurial version string to locate local packages',
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
524 )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
525 sp.set_defaults(func=publish_windows_artifacts)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42648
diff changeset
526
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
527 sp = subparsers.add_parser(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
528 'try', help='Run CI automation against a custom changeset'
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
529 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43057
diff changeset
530 sp.add_argument('-r', '--rev', default='.', help='Revision to run CI on')
43057
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
531 sp.set_defaults(func=run_try)
c5c502bd1f70 automation: add a command to submit to a Try server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43055
diff changeset
532
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
533 return parser
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
534
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
535
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
536 def main():
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
537 parser = get_parser()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
538 args = parser.parse_args()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
539
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
540 local_state_path = pathlib.Path(os.path.expanduser(args.state_path))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
541 automation = HGAutomation(local_state_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
542
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
543 if not hasattr(args, 'func'):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
544 parser.print_help()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
545 return
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
546
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
547 kwargs = dict(vars(args))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
548 del kwargs['func']
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
549 del kwargs['state_path']
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
550
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
551 args.func(automation, **kwargs)