Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/linux.py @ 51877:905bc9d0a149

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
python-compat: drop support for Python3.6 and 3.7 As discussed on the mailing list?, these are old version that seems safe to drop. Python 3.8 comes with various improvement especially regarding typing capabilities. [1] https://lists.mercurial-scm.org/pipermail/mercurial-devel/2024-July/297998.html
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Tue, 10 Sep 2024 22:26:23 +0200
parents 5c8148cd7f13
children 0e713555ecdc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # linux.py - Linux specific automation functionality
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 # no-check-code because Python 3 native.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 import os
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11 import pathlib
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 import shlex
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 import subprocess
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 import tempfile
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
16 from .ssh import exec_command
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19 # Linux distributions that are supported.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20 DISTROS = {
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 'debian9',
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
22 'debian10',
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 'ubuntu18.04',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24 'ubuntu19.04',
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25 }
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 INSTALL_PYTHONS = r'''
51877
905bc9d0a149 python-compat: drop support for Python3.6 and 3.7
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48936
diff changeset
28 PYENV3_VERSIONS="3.8.10 3.9.5 pypy3.5-7.0.0 pypy3.6-7.3.3 pypy3.7-7.3.3"
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 git clone https://github.com/pyenv/pyenv.git /hgdev/pyenv
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31 pushd /hgdev/pyenv
47207
743522fa2455 automation: install latest Python versions in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47206
diff changeset
32 git checkout 328fd42c3a2fbf14ae46dae2021a087fe27ba7e2
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 popd
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 export PYENV_ROOT="/hgdev/pyenv"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36 export PATH="$PYENV_ROOT/bin:$PATH"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37
43016
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
38 # pip 19.2.3.
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
39 PIP_SHA256=57e3643ff19f018f8a00dfaa6b7e4620e3c1a7a2171fd218425366ec006b3bfe
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
40 wget -O get-pip.py --progress dot:mega https://github.com/pypa/get-pip/raw/309a56c5fd94bd1134053a541cb4657a4e47e09d/get-pip.py
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 echo "${PIP_SHA256} get-pip.py" | sha256sum --check -
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42
43016
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
43 VIRTUALENV_SHA256=f78d81b62d3147396ac33fc9d77579ddc42cc2a98dd9ea38886f616b33bc7fb2
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
44 VIRTUALENV_TARBALL=virtualenv-16.7.5.tar.gz
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
45 wget -O ${VIRTUALENV_TARBALL} --progress dot:mega https://files.pythonhosted.org/packages/66/f0/6867af06d2e2f511e4e1d7094ff663acdebc4f15d4a0cb0fed1007395124/${VIRTUALENV_TARBALL}
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
46 echo "${VIRTUALENV_SHA256} ${VIRTUALENV_TARBALL}" | sha256sum --check -
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48 for v in ${PYENV3_VERSIONS}; do
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49 pyenv install -v ${v}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 ${PYENV_ROOT}/versions/${v}/bin/python get-pip.py
47206
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
51
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
52 case ${v} in
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
53 3.5.*)
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
54 REQUIREMENTS=requirements-py3.5.txt
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
55 ;;
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
56 pypy3.5*)
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
57 REQUIREMENTS=requirements-py3.5.txt
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
58 ;;
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
59 *)
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
60 REQUIREMENTS=requirements-py3.txt
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
61 ;;
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
62 esac
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
63
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
64 ${PYENV_ROOT}/versions/${v}/bin/pip install -r /hgdev/${REQUIREMENTS}
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 done
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
66
48936
5c8148cd7f13 automation: drop support for Python 2.7 in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48934
diff changeset
67 pyenv global ${PYENV3_VERSIONS} system
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
68 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
69 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
70 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71
47213
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
72 INSTALL_PYOXIDIZER = r'''
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
73 PYOXIDIZER_VERSION=0.16.0
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
74 PYOXIDIZER_SHA256=8875471c270312fbb934007fd30f65f1904cc0f5da6188d61c90ed2129b9f9c1
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
75 PYOXIDIZER_URL=https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F${PYOXIDIZER_VERSION}/pyoxidizer-${PYOXIDIZER_VERSION}-linux_x86_64.zip
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
76
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
77 wget -O pyoxidizer.zip --progress dot:mega ${PYOXIDIZER_URL}
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
78 echo "${PYOXIDIZER_SHA256} pyoxidizer.zip" | sha256sum --check -
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
79
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
80 unzip pyoxidizer.zip
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
81 chmod +x pyoxidizer
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
82 sudo mv pyoxidizer /usr/local/bin/pyoxidizer
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
83 '''
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
84
42718
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
85 INSTALL_RUST = r'''
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
86 RUSTUP_INIT_SHA256=a46fe67199b7bcbbde2dcbc23ae08db6f29883e260e23899a88b9073effc9076
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
87 wget -O rustup-init --progress dot:mega https://static.rust-lang.org/rustup/archive/1.18.3/x86_64-unknown-linux-gnu/rustup-init
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
88 echo "${RUSTUP_INIT_SHA256} rustup-init" | sha256sum --check -
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
89
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
90 chmod +x rustup-init
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
91 sudo -H -u hg -g hg ./rustup-init -y
47208
2df655f682a6 automation: update rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47207
diff changeset
92 sudo -H -u hg -g hg /home/hg/.cargo/bin/rustup install 1.41.1 1.52.0
42718
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
93 sudo -H -u hg -g hg /home/hg/.cargo/bin/rustup component add clippy
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
94 '''
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
95
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
96
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
97 BOOTSTRAP_VIRTUALENV = r'''
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
98 /usr/bin/virtualenv /hgdev/venv-bootstrap
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99
43016
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
100 HG_SHA256=35fc8ba5e0379c1b3affa2757e83fb0509e8ac314cbd9f1fd133cf265d16e49f
cbd94ee3a72e automation: upgrade packages in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43015
diff changeset
101 HG_TARBALL=mercurial-5.1.1.tar.gz
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
102
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
103 wget -O ${HG_TARBALL} --progress dot:mega https://www.mercurial-scm.org/release/${HG_TARBALL}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
104 echo "${HG_SHA256} ${HG_TARBALL}" | sha256sum --check -
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
105
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
106 /hgdev/venv-bootstrap/bin/pip install ${HG_TARBALL}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
107 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
108 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
109 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
110
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
111
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
112 BOOTSTRAP_DEBIAN = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
113 r'''
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
114 #!/bin/bash
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
115
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
116 set -ex
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
117
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
118 DISTRO=`grep DISTRIB_ID /etc/lsb-release | awk -F= '{{print $2}}'`
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
119 DEBIAN_VERSION=`cat /etc/debian_version`
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
120 LSB_RELEASE=`lsb_release -cs`
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
121
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
122 sudo /usr/sbin/groupadd hg
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
123 sudo /usr/sbin/groupadd docker
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
124 sudo /usr/sbin/useradd -g hg -G sudo,docker -d /home/hg -m -s /bin/bash hg
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
125 sudo mkdir /home/hg/.ssh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
126 sudo cp ~/.ssh/authorized_keys /home/hg/.ssh/authorized_keys
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
127 sudo chown -R hg:hg /home/hg/.ssh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
128 sudo chmod 700 /home/hg/.ssh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
129 sudo chmod 600 /home/hg/.ssh/authorized_keys
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
130
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
131 cat << EOF | sudo tee /etc/sudoers.d/90-hg
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
132 hg ALL=(ALL) NOPASSWD:ALL
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
133 EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
134
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
135 sudo apt-get update
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
136 sudo DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
137
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
138 # Install packages necessary to set up Docker Apt repo.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
139 sudo DEBIAN_FRONTEND=noninteractive apt-get -yq install --no-install-recommends \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
140 apt-transport-https \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
141 gnupg
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
142
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
143 cat > docker-apt-key << EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
144 -----BEGIN PGP PUBLIC KEY BLOCK-----
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
145
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
146 mQINBFit2ioBEADhWpZ8/wvZ6hUTiXOwQHXMAlaFHcPH9hAtr4F1y2+OYdbtMuth
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
147 lqqwp028AqyY+PRfVMtSYMbjuQuu5byyKR01BbqYhuS3jtqQmljZ/bJvXqnmiVXh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
148 38UuLa+z077PxyxQhu5BbqntTPQMfiyqEiU+BKbq2WmANUKQf+1AmZY/IruOXbnq
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
149 L4C1+gJ8vfmXQt99npCaxEjaNRVYfOS8QcixNzHUYnb6emjlANyEVlZzeqo7XKl7
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
150 UrwV5inawTSzWNvtjEjj4nJL8NsLwscpLPQUhTQ+7BbQXAwAmeHCUTQIvvWXqw0N
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
151 cmhh4HgeQscQHYgOJjjDVfoY5MucvglbIgCqfzAHW9jxmRL4qbMZj+b1XoePEtht
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
152 ku4bIQN1X5P07fNWzlgaRL5Z4POXDDZTlIQ/El58j9kp4bnWRCJW0lya+f8ocodo
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
153 vZZ+Doi+fy4D5ZGrL4XEcIQP/Lv5uFyf+kQtl/94VFYVJOleAv8W92KdgDkhTcTD
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
154 G7c0tIkVEKNUq48b3aQ64NOZQW7fVjfoKwEZdOqPE72Pa45jrZzvUFxSpdiNk2tZ
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
155 XYukHjlxxEgBdC/J3cMMNRE1F4NCA3ApfV1Y7/hTeOnmDuDYwr9/obA8t016Yljj
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
156 q5rdkywPf4JF8mXUW5eCN1vAFHxeg9ZWemhBtQmGxXnw9M+z6hWwc6ahmwARAQAB
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
157 tCtEb2NrZXIgUmVsZWFzZSAoQ0UgZGViKSA8ZG9ja2VyQGRvY2tlci5jb20+iQI3
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
158 BBMBCgAhBQJYrefAAhsvBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJEI2BgDwO
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
159 v82IsskP/iQZo68flDQmNvn8X5XTd6RRaUH33kXYXquT6NkHJciS7E2gTJmqvMqd
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
160 tI4mNYHCSEYxI5qrcYV5YqX9P6+Ko+vozo4nseUQLPH/ATQ4qL0Zok+1jkag3Lgk
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
161 jonyUf9bwtWxFp05HC3GMHPhhcUSexCxQLQvnFWXD2sWLKivHp2fT8QbRGeZ+d3m
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
162 6fqcd5Fu7pxsqm0EUDK5NL+nPIgYhN+auTrhgzhK1CShfGccM/wfRlei9Utz6p9P
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
163 XRKIlWnXtT4qNGZNTN0tR+NLG/6Bqd8OYBaFAUcue/w1VW6JQ2VGYZHnZu9S8LMc
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
164 FYBa5Ig9PxwGQOgq6RDKDbV+PqTQT5EFMeR1mrjckk4DQJjbxeMZbiNMG5kGECA8
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
165 g383P3elhn03WGbEEa4MNc3Z4+7c236QI3xWJfNPdUbXRaAwhy/6rTSFbzwKB0Jm
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
166 ebwzQfwjQY6f55MiI/RqDCyuPj3r3jyVRkK86pQKBAJwFHyqj9KaKXMZjfVnowLh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
167 9svIGfNbGHpucATqREvUHuQbNnqkCx8VVhtYkhDb9fEP2xBu5VvHbR+3nfVhMut5
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
168 G34Ct5RS7Jt6LIfFdtcn8CaSas/l1HbiGeRgc70X/9aYx/V/CEJv0lIe8gP6uDoW
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
169 FPIZ7d6vH+Vro6xuWEGiuMaiznap2KhZmpkgfupyFmplh0s6knymuQINBFit2ioB
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
170 EADneL9S9m4vhU3blaRjVUUyJ7b/qTjcSylvCH5XUE6R2k+ckEZjfAMZPLpO+/tF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
171 M2JIJMD4SifKuS3xck9KtZGCufGmcwiLQRzeHF7vJUKrLD5RTkNi23ydvWZgPjtx
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
172 Q+DTT1Zcn7BrQFY6FgnRoUVIxwtdw1bMY/89rsFgS5wwuMESd3Q2RYgb7EOFOpnu
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
173 w6da7WakWf4IhnF5nsNYGDVaIHzpiqCl+uTbf1epCjrOlIzkZ3Z3Yk5CM/TiFzPk
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
174 z2lLz89cpD8U+NtCsfagWWfjd2U3jDapgH+7nQnCEWpROtzaKHG6lA3pXdix5zG8
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
175 eRc6/0IbUSWvfjKxLLPfNeCS2pCL3IeEI5nothEEYdQH6szpLog79xB9dVnJyKJb
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
176 VfxXnseoYqVrRz2VVbUI5Blwm6B40E3eGVfUQWiux54DspyVMMk41Mx7QJ3iynIa
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
177 1N4ZAqVMAEruyXTRTxc9XW0tYhDMA/1GYvz0EmFpm8LzTHA6sFVtPm/ZlNCX6P1X
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
178 zJwrv7DSQKD6GGlBQUX+OeEJ8tTkkf8QTJSPUdh8P8YxDFS5EOGAvhhpMBYD42kQ
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
179 pqXjEC+XcycTvGI7impgv9PDY1RCC1zkBjKPa120rNhv/hkVk/YhuGoajoHyy4h7
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
180 ZQopdcMtpN2dgmhEegny9JCSwxfQmQ0zK0g7m6SHiKMwjwARAQABiQQ+BBgBCAAJ
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
181 BQJYrdoqAhsCAikJEI2BgDwOv82IwV0gBBkBCAAGBQJYrdoqAAoJEH6gqcPyc/zY
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
182 1WAP/2wJ+R0gE6qsce3rjaIz58PJmc8goKrir5hnElWhPgbq7cYIsW5qiFyLhkdp
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
183 YcMmhD9mRiPpQn6Ya2w3e3B8zfIVKipbMBnke/ytZ9M7qHmDCcjoiSmwEXN3wKYI
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
184 mD9VHONsl/CG1rU9Isw1jtB5g1YxuBA7M/m36XN6x2u+NtNMDB9P56yc4gfsZVES
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
185 KA9v+yY2/l45L8d/WUkUi0YXomn6hyBGI7JrBLq0CX37GEYP6O9rrKipfz73XfO7
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
186 JIGzOKZlljb/D9RX/g7nRbCn+3EtH7xnk+TK/50euEKw8SMUg147sJTcpQmv6UzZ
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
187 cM4JgL0HbHVCojV4C/plELwMddALOFeYQzTif6sMRPf+3DSj8frbInjChC3yOLy0
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
188 6br92KFom17EIj2CAcoeq7UPhi2oouYBwPxh5ytdehJkoo+sN7RIWua6P2WSmon5
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
189 U888cSylXC0+ADFdgLX9K2zrDVYUG1vo8CX0vzxFBaHwN6Px26fhIT1/hYUHQR1z
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
190 VfNDcyQmXqkOnZvvoMfz/Q0s9BhFJ/zU6AgQbIZE/hm1spsfgvtsD1frZfygXJ9f
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
191 irP+MSAI80xHSf91qSRZOj4Pl3ZJNbq4yYxv0b1pkMqeGdjdCYhLU+LZ4wbQmpCk
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
192 SVe2prlLureigXtmZfkqevRz7FrIZiu9ky8wnCAPwC7/zmS18rgP/17bOtL4/iIz
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
193 QhxAAoAMWVrGyJivSkjhSGx1uCojsWfsTAm11P7jsruIL61ZzMUVE2aM3Pmj5G+W
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
194 9AcZ58Em+1WsVnAXdUR//bMmhyr8wL/G1YO1V3JEJTRdxsSxdYa4deGBBY/Adpsw
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
195 24jxhOJR+lsJpqIUeb999+R8euDhRHG9eFO7DRu6weatUJ6suupoDTRWtr/4yGqe
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
196 dKxV3qQhNLSnaAzqW/1nA3iUB4k7kCaKZxhdhDbClf9P37qaRW467BLCVO/coL3y
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
197 Vm50dwdrNtKpMBh3ZpbB1uJvgi9mXtyBOMJ3v8RZeDzFiG8HdCtg9RvIt/AIFoHR
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
198 H3S+U79NT6i0KPzLImDfs8T7RlpyuMc4Ufs8ggyg9v3Ae6cN3eQyxcK3w0cbBwsh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
199 /nQNfsA6uu+9H7NhbehBMhYnpNZyrHzCmzyXkauwRAqoCbGCNykTRwsur9gS41TQ
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
200 M8ssD1jFheOJf3hODnkKU+HKjvMROl1DK7zdmLdNzA1cvtZH/nCC9KPj1z8QC47S
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
201 xx+dTZSx4ONAhwbS/LN3PoKtn8LPjY9NP9uDWI+TWYquS2U+KHDrBDlsgozDbs/O
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
202 jCxcpDzNmXpWQHEtHU7649OXHP7UeNST1mCUCH5qdank0V1iejF6/CfTFU4MfcrG
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
203 YT90qFF93M3v01BbxP+EIY2/9tiIPbrd
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
204 =0YYh
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
205 -----END PGP PUBLIC KEY BLOCK-----
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
206 EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
207
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
208 sudo apt-key add docker-apt-key
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
209
43013
198b51d453fe automation: use LSB_RELEASE instead of DEBIAN_VERSION
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42718
diff changeset
210 if [ "$LSB_RELEASE" = "stretch" ]; then
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
211 cat << EOF | sudo tee -a /etc/apt/sources.list
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
212 # Need backports for clang-format-6.0
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
213 deb http://deb.debian.org/debian stretch-backports main
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
214 EOF
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
215 fi
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
216
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
217 if [ "$LSB_RELEASE" = "stretch" -o "$LSB_RELEASE" = "buster" ]; then
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
218 cat << EOF | sudo tee -a /etc/apt/sources.list
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
219 # Sources are useful if we want to compile things locally.
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
220 deb-src http://deb.debian.org/debian $LSB_RELEASE main
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
221 deb-src http://security.debian.org/debian-security $LSB_RELEASE/updates main
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
222 deb-src http://deb.debian.org/debian $LSB_RELEASE-updates main
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
223 deb-src http://deb.debian.org/debian $LSB_RELEASE-backports main
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
224
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
225 deb [arch=amd64] https://download.docker.com/linux/debian $LSB_RELEASE stable
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
226 EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
227
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
228 elif [ "$DISTRO" = "Ubuntu" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
229 cat << EOF | sudo tee -a /etc/apt/sources.list
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
230 deb [arch=amd64] https://download.docker.com/linux/ubuntu $LSB_RELEASE stable
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
231 EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
232
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
233 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
234
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
235 sudo apt-get update
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
236
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
237 PACKAGES="\
43017
dffa9d3c2491 automation: install awscli and python3-boto3 packages
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43016
diff changeset
238 awscli \
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
239 btrfs-progs \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
240 build-essential \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
241 bzr \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
242 clang-format-6.0 \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
243 cvs \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
244 darcs \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
245 debhelper \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
246 devscripts \
43014
136c2536d83f automation: always install docker-ce
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43013
diff changeset
247 docker-ce \
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
248 dpkg-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
249 dstat \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
250 emacs \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
251 gettext \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
252 git \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
253 htop \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
254 iotop \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
255 jfsutils \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
256 libbz2-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
257 libexpat1-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
258 libffi-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
259 libgdbm-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
260 liblzma-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
261 libncurses5-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
262 libnss3-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
263 libreadline-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
264 libsqlite3-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
265 libssl-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
266 netbase \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
267 ntfs-3g \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
268 nvme-cli \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
269 pyflakes3 \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
270 pylint3 \
43017
dffa9d3c2491 automation: install awscli and python3-boto3 packages
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43016
diff changeset
271 python3-boto3 \
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
272 python3-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
273 python3-docutils \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
274 python3-fuzzywuzzy \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
275 python3-pygments \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
276 python3-vcr \
43413
acdd4f28f6a3 automation: install python3-venv Debian package
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43315
diff changeset
277 python3-venv \
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
278 rsync \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
279 sqlite3 \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
280 subversion \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
281 tcl-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
282 tk-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
283 tla \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
284 unzip \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
285 uuid-dev \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
286 vim \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
287 virtualenv \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
288 wget \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
289 xfsprogs \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
290 zip \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
291 zlib1g-dev"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
292
43013
198b51d453fe automation: use LSB_RELEASE instead of DEBIAN_VERSION
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42718
diff changeset
293 if [ "LSB_RELEASE" = "stretch" ]; then
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
294 PACKAGES="$PACKAGES linux-perf"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
295 elif [ "$DISTRO" = "Ubuntu" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
296 PACKAGES="$PACKAGES linux-tools-common"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
297 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
298
43020
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
299 # Monotone only available in older releases.
d1d919f679f7 automation: support and use Debian Buster by default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43017
diff changeset
300 if [ "$LSB_RELEASE" = "stretch" -o "$LSB_RELEASE" = "xenial" ]; then
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
301 PACKAGES="$PACKAGES monotone"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
302 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
303
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
304 sudo DEBIAN_FRONTEND=noninteractive apt-get -yq install --no-install-recommends $PACKAGES
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
305
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
306 # Create clang-format symlink so test harness finds it.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
307 sudo update-alternatives --install /usr/bin/clang-format clang-format \
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
308 /usr/bin/clang-format-6.0 1000
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
309
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
310 sudo mkdir /hgdev
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
311 # Will be normalized to hg:hg later.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
312 sudo chown `whoami` /hgdev
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
313
42718
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
314 {install_rust}
47213
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
315 {install_pyoxidizer}
42718
89ba81771fc9 automation: install Rust in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42716
diff changeset
316
47206
546e812a1c2d automation: create Python 3.5 variant of requirements.txt
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46349
diff changeset
317 cp requirements-*.txt /hgdev/
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
318
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
319 # Disable the pip version check because it uses the network and can
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
320 # be annoying.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
321 cat << EOF | sudo tee -a /etc/pip.conf
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
322 [global]
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
323 disable-pip-version-check = True
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
324 EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
325
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
326 {install_pythons}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
327 {bootstrap_virtualenv}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
328
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
329 /hgdev/venv-bootstrap/bin/hg clone https://www.mercurial-scm.org/repo/hg /hgdev/src
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
330
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
331 # Mark the repo as non-publishing.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
332 cat >> /hgdev/src/.hg/hgrc << EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
333 [phases]
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
334 publish = false
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
335 EOF
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
336
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
337 sudo chown -R hg:hg /hgdev
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
338 '''.lstrip()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
339 .format(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
340 install_rust=INSTALL_RUST,
47213
0de5eefbe4b2 contrib: install PyOxidizer 0.16.0
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47208
diff changeset
341 install_pyoxidizer=INSTALL_PYOXIDIZER,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
342 install_pythons=INSTALL_PYTHONS,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
343 bootstrap_virtualenv=BOOTSTRAP_VIRTUALENV,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
344 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
345 .replace('\r\n', '\n')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
346 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
347
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
348
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
349 # Prepares /hgdev for operations.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
350 PREPARE_HGDEV = '''
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
351 #!/bin/bash
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
352
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
353 set -e
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
354
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
355 FS=$1
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
356
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
357 ensure_device() {
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
358 if [ -z "${DEVICE}" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
359 echo "could not find block device to format"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
360 exit 1
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
361 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
362 }
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
363
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
364 # Determine device to partition for extra filesystem.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
365 # If only 1 volume is present, it will be the root volume and
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
366 # should be /dev/nvme0. If multiple volumes are present, the
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
367 # root volume could be nvme0 or nvme1. Use whichever one doesn't have
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
368 # a partition.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
369 if [ -e /dev/nvme1n1 ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
370 if [ -e /dev/nvme0n1p1 ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
371 DEVICE=/dev/nvme1n1
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
372 else
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
373 DEVICE=/dev/nvme0n1
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
374 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
375 else
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
376 DEVICE=
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
377 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
378
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
379 sudo mkdir /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
380
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
381 if [ "${FS}" != "default" -a "${FS}" != "tmpfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
382 ensure_device
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
383 echo "creating ${FS} filesystem on ${DEVICE}"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
384 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
385
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
386 if [ "${FS}" = "default" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
387 :
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
388
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
389 elif [ "${FS}" = "btrfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
390 sudo mkfs.btrfs ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
391 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
392
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
393 elif [ "${FS}" = "ext3" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
394 # lazy_journal_init speeds up filesystem creation at the expense of
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
395 # integrity if things crash. We are an ephemeral instance, so we don't
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
396 # care about integrity.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
397 sudo mkfs.ext3 -E lazy_journal_init=1 ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
398 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
399
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
400 elif [ "${FS}" = "ext4" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
401 sudo mkfs.ext4 -E lazy_journal_init=1 ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
402 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
403
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
404 elif [ "${FS}" = "jfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
405 sudo mkfs.jfs ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
406 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
407
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
408 elif [ "${FS}" = "tmpfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
409 echo "creating tmpfs volume in /hgwork"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
410 sudo mount -t tmpfs -o size=1024M tmpfs /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
411
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
412 elif [ "${FS}" = "xfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
413 sudo mkfs.xfs ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
414 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
415
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
416 else
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
417 echo "unsupported filesystem: ${FS}"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
418 exit 1
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
419 fi
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
420
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
421 echo "/hgwork ready"
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
422
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
423 sudo chown hg:hg /hgwork
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
424 mkdir /hgwork/tmp
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
425 chown hg:hg /hgwork/tmp
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
426
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
427 rsync -a /hgdev/src /hgwork/
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
428 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
429 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
430 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
431
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
432
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
433 HG_UPDATE_CLEAN = '''
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
434 set -ex
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
435
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
436 HG=/hgdev/venv-bootstrap/bin/hg
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
437
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
438 cd /hgwork/src
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
439 ${HG} --config extensions.purge= purge --all
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
440 ${HG} update -C $1
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
441 ${HG} log -r .
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
442 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
443 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
444 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
445
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
446
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
447 def prepare_exec_environment(ssh_client, filesystem='default'):
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
448 """Prepare an EC2 instance to execute things.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
449
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
450 The AMI has an ``/hgdev`` bootstrapped with various Python installs
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
451 and a clone of the Mercurial repo.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
452
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
453 In EC2, EBS volumes launched from snapshots have wonky performance behavior.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
454 Notably, blocks have to be copied on first access, which makes volume
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
455 I/O extremely slow on fresh volumes.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
456
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
457 Furthermore, we may want to run operations, tests, etc on alternative
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
458 filesystems so we examine behavior on different filesystems.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
459
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
460 This function is used to facilitate executing operations on alternate
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
461 volumes.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
462 """
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
463 sftp = ssh_client.open_sftp()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
464
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
465 with sftp.open('/hgdev/prepare-hgdev', 'wb') as fh:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
466 fh.write(PREPARE_HGDEV)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
467 fh.chmod(0o0777)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
468
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
469 command = 'sudo /hgdev/prepare-hgdev %s' % filesystem
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
470 chan, stdin, stdout = exec_command(ssh_client, command)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
471 stdin.close()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
472
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
473 for line in stdout:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
474 print(line, end='')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
475
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
476 res = chan.recv_exit_status()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
477
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
478 if res:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
479 raise Exception('non-0 exit code updating working directory; %d' % res)
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
480
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
481
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
482 def synchronize_hg(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
483 source_path: pathlib.Path, ec2_instance, revision: str = None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
484 ):
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
485 """Synchronize a local Mercurial source path to remote EC2 instance."""
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
486
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
487 with tempfile.TemporaryDirectory() as temp_dir:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
488 temp_dir = pathlib.Path(temp_dir)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
489
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
490 ssh_dir = temp_dir / '.ssh'
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
491 ssh_dir.mkdir()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
492 ssh_dir.chmod(0o0700)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
493
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
494 public_ip = ec2_instance.public_ip_address
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
495
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
496 ssh_config = ssh_dir / 'config'
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
497
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
498 with ssh_config.open('w', encoding='utf-8') as fh:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
499 fh.write('Host %s\n' % public_ip)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
500 fh.write(' User hg\n')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
501 fh.write(' StrictHostKeyChecking no\n')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
502 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
503 fh.write(' IdentityFile %s\n' % ec2_instance.ssh_private_key_path)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
504
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
505 if not (source_path / '.hg').is_dir():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
506 raise Exception(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
507 '%s is not a Mercurial repository; synchronization '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
508 'not yet supported' % source_path
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
509 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
510
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
511 env = dict(os.environ)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
512 env['HGPLAIN'] = '1'
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
513 env['HGENCODING'] = 'utf-8'
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
514
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
515 hg_bin = source_path / 'hg'
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
516
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
517 res = subprocess.run(
48934
d953a42b157d automation: run hg with python3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47213
diff changeset
518 ['python3', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
519 cwd=str(source_path),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
520 env=env,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
521 check=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
522 capture_output=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
523 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
524
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
525 full_revision = res.stdout.decode('ascii')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
526
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
527 args = [
48934
d953a42b157d automation: run hg with python3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47213
diff changeset
528 'python3',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
529 str(hg_bin),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
530 '--config',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
531 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
532 '--config',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
533 'ui.remotecmd=/hgdev/venv-bootstrap/bin/hg',
42658
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42651
diff changeset
534 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42651
diff changeset
535 # calculation works.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
536 'push',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
537 '-f',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
538 '-r',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
539 full_revision,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
540 '-r',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
541 'file(.hgtags)',
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
542 'ssh://%s//hgwork/src' % public_ip,
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
543 ]
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
544
42651
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
545 res = subprocess.run(args, cwd=str(source_path), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
546
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
547 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
548 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42312
diff changeset
549 res.check_returncode()
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
550
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
551 # TODO support synchronizing dirty working directory.
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
552
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
553 sftp = ec2_instance.ssh_client.open_sftp()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
554
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
555 with sftp.open('/hgdev/hgup', 'wb') as fh:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
556 fh.write(HG_UPDATE_CLEAN)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
557 fh.chmod(0o0700)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
558
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
559 chan, stdin, stdout = exec_command(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
560 ec2_instance.ssh_client, '/hgdev/hgup %s' % full_revision
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
561 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
562 stdin.close()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
563
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
564 for line in stdout:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
565 print(line, end='')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
566
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
567 res = chan.recv_exit_status()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
568
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
569 if res:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
570 raise Exception(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
571 'non-0 exit code updating working directory; %d' % res
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
572 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
573
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
574
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
575 def run_tests(ssh_client, python_version, test_flags=None):
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
576 """Run tests on a remote Linux machine via an SSH client."""
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
577 test_flags = test_flags or []
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
578
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
579 print('running tests')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
580
48936
5c8148cd7f13 automation: drop support for Python 2.7 in Linux environment
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48934
diff changeset
581 if python_version == 'system3':
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
582 python = '/usr/bin/python3'
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
583 elif python_version.startswith('pypy'):
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
584 python = '/hgdev/pyenv/shims/%s' % python_version
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
585 else:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
586 python = '/hgdev/pyenv/shims/python%s' % python_version
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
587
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
588 test_flags = ' '.join(shlex.quote(a) for a in test_flags)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
589
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
590 command = (
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
591 '/bin/sh -c "export TMPDIR=/hgwork/tmp; '
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
592 'cd /hgwork/src/tests && %s run-tests.py %s"' % (python, test_flags)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43020
diff changeset
593 )
42312
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
594
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
595 chan, stdin, stdout = exec_command(ssh_client, command)
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
596
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
597 stdin.close()
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
598
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
599 for line in stdout:
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
600 print(line, end='')
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
601
65b3ef162b39 automation: initial support for running Linux tests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
602 return chan.recv_exit_status()