mercurial-scm/hg-stable: contrib/automation/hgautomation/linux.py annotate

annotate contrib/automation/hgautomation/linux.py @ 43235:252ec4dbc1b4

automation: install Python 3.8.0 Python 3.8 has been released. Let's install it instead of the RC / development version. Differential Revision: https://phab.mercurial-scm.org/D7114

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Tue, 15 Oct 2019 19:13:47 -0700
parents	2372284d9457
children	388ada1c07c6

rev	line source
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # linux.py - Linux specific automation functionality
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # no-check-code because Python 3 native.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 import os
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 import pathlib
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 import shlex
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 import subprocess
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import tempfile
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	16 from .ssh import exec_command
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	18
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	19 # Linux distributions that are supported.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20 DISTROS = {
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 'debian9',
43020 d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	22 'debian10',
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 'ubuntu18.04',
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 'ubuntu19.04',
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 }
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 INSTALL_PYTHONS = r'''
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 PYENV2_VERSIONS="2.7.16 pypy2.7-7.1.1"
43235 252ec4dbc1b4 automation: install Python 3.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	29 PYENV3_VERSIONS="3.5.7 3.6.9 3.7.4 3.8.0 pypy3.5-7.0.0 pypy3.6-7.1.1"
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31 git clone https://github.com/pyenv/pyenv.git /hgdev/pyenv
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 pushd /hgdev/pyenv
43235 252ec4dbc1b4 automation: install Python 3.8.0 Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	33 git checkout d6d6bc8bb08bcdcbf4eb79509aa7061011ade1c4
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 popd
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 export PYENV_ROOT="/hgdev/pyenv"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 export PATH="$PYENV_ROOT/bin:$PATH"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38
43016 cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	39 # pip 19.2.3.
cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	40 PIP_SHA256=57e3643ff19f018f8a00dfaa6b7e4620e3c1a7a2171fd218425366ec006b3bfe
cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	41 wget -O get-pip.py --progress dot:mega https://github.com/pypa/get-pip/raw/309a56c5fd94bd1134053a541cb4657a4e47e09d/get-pip.py
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 echo "${PIP_SHA256} get-pip.py" \| sha256sum --check -
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	43
43016 cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	44 VIRTUALENV_SHA256=f78d81b62d3147396ac33fc9d77579ddc42cc2a98dd9ea38886f616b33bc7fb2
cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	45 VIRTUALENV_TARBALL=virtualenv-16.7.5.tar.gz
cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	46 wget -O ${VIRTUALENV_TARBALL} --progress dot:mega https://files.pythonhosted.org/packages/66/f0/6867af06d2e2f511e4e1d7094ff663acdebc4f15d4a0cb0fed1007395124/${VIRTUALENV_TARBALL}
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	47 echo "${VIRTUALENV_SHA256} ${VIRTUALENV_TARBALL}" \| sha256sum --check -
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	48
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	49 for v in ${PYENV2_VERSIONS}; do
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	50 pyenv install -v ${v}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51 ${PYENV_ROOT}/versions/${v}/bin/python get-pip.py
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 ${PYENV_ROOT}/versions/${v}/bin/pip install ${VIRTUALENV_TARBALL}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	53 ${PYENV_ROOT}/versions/${v}/bin/pip install -r /hgdev/requirements-py2.txt
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 done
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	56 for v in ${PYENV3_VERSIONS}; do
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 pyenv install -v ${v}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 ${PYENV_ROOT}/versions/${v}/bin/python get-pip.py
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59 ${PYENV_ROOT}/versions/${v}/bin/pip install -r /hgdev/requirements-py3.txt
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	60 done
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	62 pyenv global ${PYENV2_VERSIONS} ${PYENV3_VERSIONS} system
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	63 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	64 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	65 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	66
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	67
42718 89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	68 INSTALL_RUST = r'''
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	69 RUSTUP_INIT_SHA256=a46fe67199b7bcbbde2dcbc23ae08db6f29883e260e23899a88b9073effc9076
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	70 wget -O rustup-init --progress dot:mega https://static.rust-lang.org/rustup/archive/1.18.3/x86_64-unknown-linux-gnu/rustup-init
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	71 echo "${RUSTUP_INIT_SHA256} rustup-init" \| sha256sum --check -
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	72
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	73 chmod +x rustup-init
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	74 sudo -H -u hg -g hg ./rustup-init -y
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	75 sudo -H -u hg -g hg /home/hg/.cargo/bin/rustup install 1.31.1 1.34.2
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	76 sudo -H -u hg -g hg /home/hg/.cargo/bin/rustup component add clippy
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	77 '''
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	78
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	79
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 BOOTSTRAP_VIRTUALENV = r'''
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 /usr/bin/virtualenv /hgdev/venv-bootstrap
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82
43016 cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	83 HG_SHA256=35fc8ba5e0379c1b3affa2757e83fb0509e8ac314cbd9f1fd133cf265d16e49f
cbd94ee3a72e automation: upgrade packages in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 43015 diff changeset	84 HG_TARBALL=mercurial-5.1.1.tar.gz
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	86 wget -O ${HG_TARBALL} --progress dot:mega https://www.mercurial-scm.org/release/${HG_TARBALL}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	87 echo "${HG_SHA256} ${HG_TARBALL}" \| sha256sum --check -
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	88
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	89 /hgdev/venv-bootstrap/bin/pip install ${HG_TARBALL}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	90 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	91 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	92 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	93
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	94
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	95 BOOTSTRAP_DEBIAN = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	96 r'''
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	97 #!/bin/bash
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	98
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99 set -ex
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	100
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 DISTRO=`grep DISTRIB_ID /etc/lsb-release \| awk -F= '{{print $2}}'`
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	102 DEBIAN_VERSION=`cat /etc/debian_version`
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	103 LSB_RELEASE=`lsb_release -cs`
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	104
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	105 sudo /usr/sbin/groupadd hg
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	106 sudo /usr/sbin/groupadd docker
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	107 sudo /usr/sbin/useradd -g hg -G sudo,docker -d /home/hg -m -s /bin/bash hg
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	108 sudo mkdir /home/hg/.ssh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	109 sudo cp ~/.ssh/authorized_keys /home/hg/.ssh/authorized_keys
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110 sudo chown -R hg:hg /home/hg/.ssh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	111 sudo chmod 700 /home/hg/.ssh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	112 sudo chmod 600 /home/hg/.ssh/authorized_keys
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	113
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	114 cat << EOF \| sudo tee /etc/sudoers.d/90-hg
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	115 hg ALL=(ALL) NOPASSWD:ALL
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	116 EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	117
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	118 sudo apt-get update
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	119 sudo DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	120
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	121 # Install packages necessary to set up Docker Apt repo.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	122 sudo DEBIAN_FRONTEND=noninteractive apt-get -yq install --no-install-recommends \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123 apt-transport-https \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	124 gnupg
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	125
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	126 cat > docker-apt-key << EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	127 -----BEGIN PGP PUBLIC KEY BLOCK-----
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	128
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	129 mQINBFit2ioBEADhWpZ8/wvZ6hUTiXOwQHXMAlaFHcPH9hAtr4F1y2+OYdbtMuth
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	130 lqqwp028AqyY+PRfVMtSYMbjuQuu5byyKR01BbqYhuS3jtqQmljZ/bJvXqnmiVXh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	131 38UuLa+z077PxyxQhu5BbqntTPQMfiyqEiU+BKbq2WmANUKQf+1AmZY/IruOXbnq
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	132 L4C1+gJ8vfmXQt99npCaxEjaNRVYfOS8QcixNzHUYnb6emjlANyEVlZzeqo7XKl7
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133 UrwV5inawTSzWNvtjEjj4nJL8NsLwscpLPQUhTQ+7BbQXAwAmeHCUTQIvvWXqw0N
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	134 cmhh4HgeQscQHYgOJjjDVfoY5MucvglbIgCqfzAHW9jxmRL4qbMZj+b1XoePEtht
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135 ku4bIQN1X5P07fNWzlgaRL5Z4POXDDZTlIQ/El58j9kp4bnWRCJW0lya+f8ocodo
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136 vZZ+Doi+fy4D5ZGrL4XEcIQP/Lv5uFyf+kQtl/94VFYVJOleAv8W92KdgDkhTcTD
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 G7c0tIkVEKNUq48b3aQ64NOZQW7fVjfoKwEZdOqPE72Pa45jrZzvUFxSpdiNk2tZ
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138 XYukHjlxxEgBdC/J3cMMNRE1F4NCA3ApfV1Y7/hTeOnmDuDYwr9/obA8t016Yljj
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139 q5rdkywPf4JF8mXUW5eCN1vAFHxeg9ZWemhBtQmGxXnw9M+z6hWwc6ahmwARAQAB
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140 tCtEb2NrZXIgUmVsZWFzZSAoQ0UgZGViKSA8ZG9ja2VyQGRvY2tlci5jb20+iQI3
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141 BBMBCgAhBQJYrefAAhsvBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJEI2BgDwO
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 v82IsskP/iQZo68flDQmNvn8X5XTd6RRaUH33kXYXquT6NkHJciS7E2gTJmqvMqd
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 tI4mNYHCSEYxI5qrcYV5YqX9P6+Ko+vozo4nseUQLPH/ATQ4qL0Zok+1jkag3Lgk
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144 jonyUf9bwtWxFp05HC3GMHPhhcUSexCxQLQvnFWXD2sWLKivHp2fT8QbRGeZ+d3m
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145 6fqcd5Fu7pxsqm0EUDK5NL+nPIgYhN+auTrhgzhK1CShfGccM/wfRlei9Utz6p9P
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 XRKIlWnXtT4qNGZNTN0tR+NLG/6Bqd8OYBaFAUcue/w1VW6JQ2VGYZHnZu9S8LMc
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	147 FYBa5Ig9PxwGQOgq6RDKDbV+PqTQT5EFMeR1mrjckk4DQJjbxeMZbiNMG5kGECA8
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	148 g383P3elhn03WGbEEa4MNc3Z4+7c236QI3xWJfNPdUbXRaAwhy/6rTSFbzwKB0Jm
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	149 ebwzQfwjQY6f55MiI/RqDCyuPj3r3jyVRkK86pQKBAJwFHyqj9KaKXMZjfVnowLh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	150 9svIGfNbGHpucATqREvUHuQbNnqkCx8VVhtYkhDb9fEP2xBu5VvHbR+3nfVhMut5
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	151 G34Ct5RS7Jt6LIfFdtcn8CaSas/l1HbiGeRgc70X/9aYx/V/CEJv0lIe8gP6uDoW
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	152 FPIZ7d6vH+Vro6xuWEGiuMaiznap2KhZmpkgfupyFmplh0s6knymuQINBFit2ioB
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	153 EADneL9S9m4vhU3blaRjVUUyJ7b/qTjcSylvCH5XUE6R2k+ckEZjfAMZPLpO+/tF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154 M2JIJMD4SifKuS3xck9KtZGCufGmcwiLQRzeHF7vJUKrLD5RTkNi23ydvWZgPjtx
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155 Q+DTT1Zcn7BrQFY6FgnRoUVIxwtdw1bMY/89rsFgS5wwuMESd3Q2RYgb7EOFOpnu
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 w6da7WakWf4IhnF5nsNYGDVaIHzpiqCl+uTbf1epCjrOlIzkZ3Z3Yk5CM/TiFzPk
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	157 z2lLz89cpD8U+NtCsfagWWfjd2U3jDapgH+7nQnCEWpROtzaKHG6lA3pXdix5zG8
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	158 eRc6/0IbUSWvfjKxLLPfNeCS2pCL3IeEI5nothEEYdQH6szpLog79xB9dVnJyKJb
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	159 VfxXnseoYqVrRz2VVbUI5Blwm6B40E3eGVfUQWiux54DspyVMMk41Mx7QJ3iynIa
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	160 1N4ZAqVMAEruyXTRTxc9XW0tYhDMA/1GYvz0EmFpm8LzTHA6sFVtPm/ZlNCX6P1X
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	161 zJwrv7DSQKD6GGlBQUX+OeEJ8tTkkf8QTJSPUdh8P8YxDFS5EOGAvhhpMBYD42kQ
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162 pqXjEC+XcycTvGI7impgv9PDY1RCC1zkBjKPa120rNhv/hkVk/YhuGoajoHyy4h7
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163 ZQopdcMtpN2dgmhEegny9JCSwxfQmQ0zK0g7m6SHiKMwjwARAQABiQQ+BBgBCAAJ
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	164 BQJYrdoqAhsCAikJEI2BgDwOv82IwV0gBBkBCAAGBQJYrdoqAAoJEH6gqcPyc/zY
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165 1WAP/2wJ+R0gE6qsce3rjaIz58PJmc8goKrir5hnElWhPgbq7cYIsW5qiFyLhkdp
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 YcMmhD9mRiPpQn6Ya2w3e3B8zfIVKipbMBnke/ytZ9M7qHmDCcjoiSmwEXN3wKYI
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167 mD9VHONsl/CG1rU9Isw1jtB5g1YxuBA7M/m36XN6x2u+NtNMDB9P56yc4gfsZVES
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 KA9v+yY2/l45L8d/WUkUi0YXomn6hyBGI7JrBLq0CX37GEYP6O9rrKipfz73XfO7
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	169 JIGzOKZlljb/D9RX/g7nRbCn+3EtH7xnk+TK/50euEKw8SMUg147sJTcpQmv6UzZ
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170 cM4JgL0HbHVCojV4C/plELwMddALOFeYQzTif6sMRPf+3DSj8frbInjChC3yOLy0
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171 6br92KFom17EIj2CAcoeq7UPhi2oouYBwPxh5ytdehJkoo+sN7RIWua6P2WSmon5
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172 U888cSylXC0+ADFdgLX9K2zrDVYUG1vo8CX0vzxFBaHwN6Px26fhIT1/hYUHQR1z
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 VfNDcyQmXqkOnZvvoMfz/Q0s9BhFJ/zU6AgQbIZE/hm1spsfgvtsD1frZfygXJ9f
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 irP+MSAI80xHSf91qSRZOj4Pl3ZJNbq4yYxv0b1pkMqeGdjdCYhLU+LZ4wbQmpCk
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 SVe2prlLureigXtmZfkqevRz7FrIZiu9ky8wnCAPwC7/zmS18rgP/17bOtL4/iIz
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176 QhxAAoAMWVrGyJivSkjhSGx1uCojsWfsTAm11P7jsruIL61ZzMUVE2aM3Pmj5G+W
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 9AcZ58Em+1WsVnAXdUR//bMmhyr8wL/G1YO1V3JEJTRdxsSxdYa4deGBBY/Adpsw
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178 24jxhOJR+lsJpqIUeb999+R8euDhRHG9eFO7DRu6weatUJ6suupoDTRWtr/4yGqe
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 dKxV3qQhNLSnaAzqW/1nA3iUB4k7kCaKZxhdhDbClf9P37qaRW467BLCVO/coL3y
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180 Vm50dwdrNtKpMBh3ZpbB1uJvgi9mXtyBOMJ3v8RZeDzFiG8HdCtg9RvIt/AIFoHR
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	181 H3S+U79NT6i0KPzLImDfs8T7RlpyuMc4Ufs8ggyg9v3Ae6cN3eQyxcK3w0cbBwsh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	182 /nQNfsA6uu+9H7NhbehBMhYnpNZyrHzCmzyXkauwRAqoCbGCNykTRwsur9gS41TQ
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	183 M8ssD1jFheOJf3hODnkKU+HKjvMROl1DK7zdmLdNzA1cvtZH/nCC9KPj1z8QC47S
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184 xx+dTZSx4ONAhwbS/LN3PoKtn8LPjY9NP9uDWI+TWYquS2U+KHDrBDlsgozDbs/O
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185 jCxcpDzNmXpWQHEtHU7649OXHP7UeNST1mCUCH5qdank0V1iejF6/CfTFU4MfcrG
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	186 YT90qFF93M3v01BbxP+EIY2/9tiIPbrd
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 =0YYh
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 -----END PGP PUBLIC KEY BLOCK-----
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189 EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191 sudo apt-key add docker-apt-key
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192
43013 198b51d453fe automation: use LSB_RELEASE instead of DEBIAN_VERSION Gregory Szorc <gregory.szorc@gmail.com> parents: 42718 diff changeset	193 if [ "$LSB_RELEASE" = "stretch" ]; then
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 cat << EOF \| sudo tee -a /etc/apt/sources.list
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	195 # Need backports for clang-format-6.0
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	196 deb http://deb.debian.org/debian stretch-backports main
43020 d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	197 EOF
d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	198 fi
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	199
43020 d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	200 if [ "$LSB_RELEASE" = "stretch" -o "$LSB_RELEASE" = "buster" ]; then
d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	201 cat << EOF \| sudo tee -a /etc/apt/sources.list
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	202 # Sources are useful if we want to compile things locally.
43020 d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	203 deb-src http://deb.debian.org/debian $LSB_RELEASE main
d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	204 deb-src http://security.debian.org/debian-security $LSB_RELEASE/updates main
d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	205 deb-src http://deb.debian.org/debian $LSB_RELEASE-updates main
d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	206 deb-src http://deb.debian.org/debian $LSB_RELEASE-backports main
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207
43020 d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	208 deb [arch=amd64] https://download.docker.com/linux/debian $LSB_RELEASE stable
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	209 EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 elif [ "$DISTRO" = "Ubuntu" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	212 cat << EOF \| sudo tee -a /etc/apt/sources.list
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 deb [arch=amd64] https://download.docker.com/linux/ubuntu $LSB_RELEASE stable
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214 EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218 sudo apt-get update
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220 PACKAGES="\
43017 dffa9d3c2491 automation: install awscli and python3-boto3 packages Gregory Szorc <gregory.szorc@gmail.com> parents: 43016 diff changeset	221 awscli \
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 btrfs-progs \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223 build-essential \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	224 bzr \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	225 clang-format-6.0 \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226 cvs \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227 darcs \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	228 debhelper \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229 devscripts \
43014 136c2536d83f automation: always install docker-ce Gregory Szorc <gregory.szorc@gmail.com> parents: 43013 diff changeset	230 docker-ce \
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231 dpkg-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	232 dstat \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233 emacs \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 gettext \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235 git \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 htop \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237 iotop \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238 jfsutils \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239 libbz2-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	240 libexpat1-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241 libffi-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	242 libgdbm-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	243 liblzma-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	244 libncurses5-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	245 libnss3-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	246 libreadline-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	247 libsqlite3-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248 libssl-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	249 netbase \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250 ntfs-3g \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	251 nvme-cli \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252 pyflakes \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253 pyflakes3 \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254 pylint \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255 pylint3 \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256 python-all-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 python-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258 python-docutils \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259 python-fuzzywuzzy \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	260 python-pygments \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 python-subversion \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 python-vcr \
43017 dffa9d3c2491 automation: install awscli and python3-boto3 packages Gregory Szorc <gregory.szorc@gmail.com> parents: 43016 diff changeset	263 python3-boto3 \
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264 python3-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	265 python3-docutils \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	266 python3-fuzzywuzzy \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	267 python3-pygments \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	268 python3-vcr \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	269 rsync \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	270 sqlite3 \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	271 subversion \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	272 tcl-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	273 tk-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 tla \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275 unzip \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	276 uuid-dev \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	277 vim \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	278 virtualenv \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	279 wget \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	280 xfsprogs \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	281 zip \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	282 zlib1g-dev"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	283
43013 198b51d453fe automation: use LSB_RELEASE instead of DEBIAN_VERSION Gregory Szorc <gregory.szorc@gmail.com> parents: 42718 diff changeset	284 if [ "LSB_RELEASE" = "stretch" ]; then
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	285 PACKAGES="$PACKAGES linux-perf"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	286 elif [ "$DISTRO" = "Ubuntu" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	287 PACKAGES="$PACKAGES linux-tools-common"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	288 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	289
43020 d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	290 # Monotone only available in older releases.
d1d919f679f7 automation: support and use Debian Buster by default Gregory Szorc <gregory.szorc@gmail.com> parents: 43017 diff changeset	291 if [ "$LSB_RELEASE" = "stretch" -o "$LSB_RELEASE" = "xenial" ]; then
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292 PACKAGES="$PACKAGES monotone"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295 sudo DEBIAN_FRONTEND=noninteractive apt-get -yq install --no-install-recommends $PACKAGES
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	296
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297 # Create clang-format symlink so test harness finds it.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	298 sudo update-alternatives --install /usr/bin/clang-format clang-format \
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299 /usr/bin/clang-format-6.0 1000
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	300
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301 sudo mkdir /hgdev
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	302 # Will be normalized to hg:hg later.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	303 sudo chown `whoami` /hgdev
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304
42718 89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	305 {install_rust}
89ba81771fc9 automation: install Rust in Linux environment Gregory Szorc <gregory.szorc@gmail.com> parents: 42716 diff changeset	306
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307 cp requirements-py2.txt /hgdev/requirements-py2.txt
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 cp requirements-py3.txt /hgdev/requirements-py3.txt
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	309
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	310 # Disable the pip version check because it uses the network and can
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	311 # be annoying.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312 cat << EOF \| sudo tee -a /etc/pip.conf
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313 [global]
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314 disable-pip-version-check = True
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	315 EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	316
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	317 {install_pythons}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318 {bootstrap_virtualenv}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	319
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	320 /hgdev/venv-bootstrap/bin/hg clone https://www.mercurial-scm.org/repo/hg /hgdev/src
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	321
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	322 # Mark the repo as non-publishing.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323 cat >> /hgdev/src/.hg/hgrc << EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	324 [phases]
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	325 publish = false
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	326 EOF
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	328 sudo chown -R hg:hg /hgdev
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	329 '''.lstrip()
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	330 .format(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	331 install_rust=INSTALL_RUST,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	332 install_pythons=INSTALL_PYTHONS,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	333 bootstrap_virtualenv=BOOTSTRAP_VIRTUALENV,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	334 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	335 .replace('\r\n', '\n')
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	336 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 # Prepares /hgdev for operations.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340 PREPARE_HGDEV = '''
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	341 #!/bin/bash
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	342
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	343 set -e
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	344
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	345 FS=$1
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	346
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347 ensure_device() {
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 if [ -z "${DEVICE}" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 echo "could not find block device to format"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 exit 1
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352 }
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 # Determine device to partition for extra filesystem.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355 # If only 1 volume is present, it will be the root volume and
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	356 # should be /dev/nvme0. If multiple volumes are present, the
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	357 # root volume could be nvme0 or nvme1. Use whichever one doesn't have
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	358 # a partition.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	359 if [ -e /dev/nvme1n1 ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	360 if [ -e /dev/nvme0n1p1 ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	361 DEVICE=/dev/nvme1n1
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	362 else
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	363 DEVICE=/dev/nvme0n1
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365 else
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	366 DEVICE=
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	367 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	368
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	369 sudo mkdir /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	370
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	371 if [ "${FS}" != "default" -a "${FS}" != "tmpfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	372 ensure_device
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	373 echo "creating ${FS} filesystem on ${DEVICE}"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	374 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	375
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	376 if [ "${FS}" = "default" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	377 :
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	378
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	379 elif [ "${FS}" = "btrfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	380 sudo mkfs.btrfs ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	381 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	382
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	383 elif [ "${FS}" = "ext3" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	384 # lazy_journal_init speeds up filesystem creation at the expense of
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	385 # integrity if things crash. We are an ephemeral instance, so we don't
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	386 # care about integrity.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	387 sudo mkfs.ext3 -E lazy_journal_init=1 ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	388 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	389
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	390 elif [ "${FS}" = "ext4" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	391 sudo mkfs.ext4 -E lazy_journal_init=1 ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	392 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	393
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	394 elif [ "${FS}" = "jfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	395 sudo mkfs.jfs ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	396 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	397
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	398 elif [ "${FS}" = "tmpfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	399 echo "creating tmpfs volume in /hgwork"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	400 sudo mount -t tmpfs -o size=1024M tmpfs /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	401
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	402 elif [ "${FS}" = "xfs" ]; then
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	403 sudo mkfs.xfs ${DEVICE}
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	404 sudo mount ${DEVICE} /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	405
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	406 else
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	407 echo "unsupported filesystem: ${FS}"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	408 exit 1
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	409 fi
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	410
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	411 echo "/hgwork ready"
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	412
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	413 sudo chown hg:hg /hgwork
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	414 mkdir /hgwork/tmp
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	415 chown hg:hg /hgwork/tmp
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	416
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	417 rsync -a /hgdev/src /hgwork/
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	418 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	419 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	420 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	421
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	422
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	423 HG_UPDATE_CLEAN = '''
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	424 set -ex
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	425
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	426 HG=/hgdev/venv-bootstrap/bin/hg
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	427
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	428 cd /hgwork/src
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	429 ${HG} --config extensions.purge= purge --all
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	430 ${HG} update -C $1
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	431 ${HG} log -r .
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	432 '''.lstrip().replace(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	433 '\r\n', '\n'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	434 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	435
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	436
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	437 def prepare_exec_environment(ssh_client, filesystem='default'):
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	438 """Prepare an EC2 instance to execute things.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	439
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	440 The AMI has an ``/hgdev`` bootstrapped with various Python installs
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	441 and a clone of the Mercurial repo.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	442
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	443 In EC2, EBS volumes launched from snapshots have wonky performance behavior.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	444 Notably, blocks have to be copied on first access, which makes volume
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	445 I/O extremely slow on fresh volumes.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	446
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	447 Furthermore, we may want to run operations, tests, etc on alternative
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	448 filesystems so we examine behavior on different filesystems.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	449
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	450 This function is used to facilitate executing operations on alternate
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	451 volumes.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	452 """
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	453 sftp = ssh_client.open_sftp()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	454
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	455 with sftp.open('/hgdev/prepare-hgdev', 'wb') as fh:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	456 fh.write(PREPARE_HGDEV)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	457 fh.chmod(0o0777)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	458
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	459 command = 'sudo /hgdev/prepare-hgdev %s' % filesystem
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	460 chan, stdin, stdout = exec_command(ssh_client, command)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	461 stdin.close()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	462
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	463 for line in stdout:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	464 print(line, end='')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	465
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	466 res = chan.recv_exit_status()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	467
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	468 if res:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	469 raise Exception('non-0 exit code updating working directory; %d' % res)
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	470
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	471
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	472 def synchronize_hg(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	473 source_path: pathlib.Path, ec2_instance, revision: str = None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	474 ):
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	475 """Synchronize a local Mercurial source path to remote EC2 instance."""
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	476
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	477 with tempfile.TemporaryDirectory() as temp_dir:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	478 temp_dir = pathlib.Path(temp_dir)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	479
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	480 ssh_dir = temp_dir / '.ssh'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	481 ssh_dir.mkdir()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	482 ssh_dir.chmod(0o0700)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	483
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	484 public_ip = ec2_instance.public_ip_address
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	485
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	486 ssh_config = ssh_dir / 'config'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	487
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	488 with ssh_config.open('w', encoding='utf-8') as fh:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	489 fh.write('Host %s\n' % public_ip)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	490 fh.write(' User hg\n')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	491 fh.write(' StrictHostKeyChecking no\n')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	492 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	493 fh.write(' IdentityFile %s\n' % ec2_instance.ssh_private_key_path)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	494
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	495 if not (source_path / '.hg').is_dir():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	496 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	497 '%s is not a Mercurial repository; synchronization '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	498 'not yet supported' % source_path
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	499 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	500
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	501 env = dict(os.environ)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	502 env['HGPLAIN'] = '1'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	503 env['HGENCODING'] = 'utf-8'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	504
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	505 hg_bin = source_path / 'hg'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	506
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	507 res = subprocess.run(
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	508 ['python2.7', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	509 cwd=str(source_path),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	510 env=env,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	511 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	512 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	513 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	514
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	515 full_revision = res.stdout.decode('ascii')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	516
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	517 args = [
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	518 'python2.7',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	519 str(hg_bin),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	520 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	521 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	522 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	523 'ui.remotecmd=/hgdev/venv-bootstrap/bin/hg',
42658 9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	524 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	525 # calculation works.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	526 'push',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	527 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	528 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	529 full_revision,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	530 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	531 'file(.hgtags)',
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	532 'ssh://%s//hgwork/src' % public_ip,
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	533 ]
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	534
42651 24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42312 diff changeset	535 res = subprocess.run(args, cwd=str(source_path), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42312 diff changeset	536
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42312 diff changeset	537 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42312 diff changeset	538 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42312 diff changeset	539 res.check_returncode()
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	540
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	541 # TODO support synchronizing dirty working directory.
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	542
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	543 sftp = ec2_instance.ssh_client.open_sftp()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	544
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	545 with sftp.open('/hgdev/hgup', 'wb') as fh:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	546 fh.write(HG_UPDATE_CLEAN)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	547 fh.chmod(0o0700)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	548
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	549 chan, stdin, stdout = exec_command(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	550 ec2_instance.ssh_client, '/hgdev/hgup %s' % full_revision
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	551 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	552 stdin.close()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	553
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	554 for line in stdout:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	555 print(line, end='')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	556
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	557 res = chan.recv_exit_status()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	558
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	559 if res:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	560 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	561 'non-0 exit code updating working directory; %d' % res
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	562 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	563
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	564
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	565 def run_tests(ssh_client, python_version, test_flags=None):
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	566 """Run tests on a remote Linux machine via an SSH client."""
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	567 test_flags = test_flags or []
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	568
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	569 print('running tests')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	570
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	571 if python_version == 'system2':
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	572 python = '/usr/bin/python2'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	573 elif python_version == 'system3':
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	574 python = '/usr/bin/python3'
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	575 elif python_version.startswith('pypy'):
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	576 python = '/hgdev/pyenv/shims/%s' % python_version
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	577 else:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	578 python = '/hgdev/pyenv/shims/python%s' % python_version
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	579
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	580 test_flags = ' '.join(shlex.quote(a) for a in test_flags)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	581
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	582 command = (
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	583 '/bin/sh -c "export TMPDIR=/hgwork/tmp; '
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	584 'cd /hgwork/src/tests && %s run-tests.py %s"' % (python, test_flags)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43020 diff changeset	585 )
42312 65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	586
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	587 chan, stdin, stdout = exec_command(ssh_client, command)
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	588
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	589 stdin.close()
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	590
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	591 for line in stdout:
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	592 print(line, end='')
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	593
65b3ef162b39 automation: initial support for running Linux tests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	594 return chan.recv_exit_status()

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/linux.py @ 43235:252ec4dbc1b4