mercurial-scm/hg-stable: contrib/automation/hgautomation/windows.py annotate

annotate contrib/automation/hgautomation/windows.py @ 49404:e4e33b779fa2 stable

automation: set PATH when building on Windows Sometime in the 6.2 release cycle the Windows building automation broke. Building the wheel and even PyOxidizer based installers now fails with: ``` Exception: PowerShell execution failed: error: subprocess-exited-with-error Getting requirements to build wheel did not run successfully. exit code: 1 [1 lines of output] Unable to find a working hg binary to extract the version from the repository tags [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ``` I have a hunch this is a regression from upgrading pip in 1c00777702da, but I haven't verified this. It may not be, as PyOxidizer has its own bundled Python/pip. So maybe it is something in `setup.py`.

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Tue, 19 Jul 2022 18:32:40 -0700
parents	4561ec90d3c1
children

rev	line source
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	10 import datetime
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 import os
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	12 import paramiko
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 import pathlib
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import re
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import subprocess
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 import tempfile
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	18 from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	19 from .winrm import run_powershell
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 }
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29 Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	43 hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	47 BUILD_INNO_PYTHON3 = r'''
49404 e4e33b779fa2 automation: set PATH when building on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 48938 diff changeset	48 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	49 $Env:RUSTUP_HOME = "C:\hgdev\rustup"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	50 $Env:CARGO_HOME = "C:\hgdev\cargo"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	51 Set-Location C:\hgdev\src
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	52 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --pyoxidizer-target {pyoxidizer_target} --version {version}
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	53 if ($LASTEXITCODE -ne 0) {{
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	54 throw "process exited non-0: $LASTEXITCODE"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	55 }}
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	56 '''
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	57
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59 BUILD_WHEEL = r'''
49404 e4e33b779fa2 automation: set PATH when building on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 48938 diff changeset	60 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61 Set-Location C:\hgdev\src
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	62 C:\hgdev\python{python_version}-{arch}\python.exe -m pip wheel --wheel-dir dist .
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	66 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	67
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	68 BUILD_WIX_PYTHON3 = r'''
49404 e4e33b779fa2 automation: set PATH when building on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 48938 diff changeset	69 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	70 $Env:RUSTUP_HOME = "C:\hgdev\rustup"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	71 $Env:CARGO_HOME = "C:\hgdev\cargo"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	72 Set-Location C:\hgdev\src
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	73 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py wix --pyoxidizer-target {pyoxidizer_target} --version {version}
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	74 if ($LASTEXITCODE -ne 0) {{
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	75 throw "process exited non-0: $LASTEXITCODE"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	76 }}
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	77 '''
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	78
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	86
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	87
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	88 WHEEL_FILENAME_PYTHON37_X86 = 'mercurial-{version}-cp37-cp37m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	89 WHEEL_FILENAME_PYTHON37_X64 = 'mercurial-{version}-cp37-cp37m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	90 WHEEL_FILENAME_PYTHON38_X86 = 'mercurial-{version}-cp38-cp38-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	91 WHEEL_FILENAME_PYTHON38_X64 = 'mercurial-{version}-cp38-cp38-win_amd64.whl'
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	92 WHEEL_FILENAME_PYTHON39_X86 = 'mercurial-{version}-cp39-cp39-win32.whl'
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	93 WHEEL_FILENAME_PYTHON39_X64 = 'mercurial-{version}-cp39-cp39-win_amd64.whl'
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	94 WHEEL_FILENAME_PYTHON310_X86 = 'mercurial-{version}-cp310-cp310-win32.whl'
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	95 WHEEL_FILENAME_PYTHON310_X64 = 'mercurial-{version}-cp310-cp310-win_amd64.whl'
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	96
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	97 EXE_FILENAME_PYTHON3_X86 = 'Mercurial-{version}-x86.exe'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	98 EXE_FILENAME_PYTHON3_X64 = 'Mercurial-{version}-x64.exe'
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	99
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	100 MSI_FILENAME_PYTHON3_X86 = 'mercurial-{version}-x86.msi'
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	101 MSI_FILENAME_PYTHON3_X64 = 'mercurial-{version}-x64.msi'
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	102
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	103 MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	104
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	105 X86_USER_AGENT_PATTERN = '.Windows.'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	106 X64_USER_AGENT_PATTERN = '.Windows.(WOW\|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	107
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	108 # TODO remove Python version once Python 2 is dropped.
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	109 EXE_PYTHON3_X86_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	110 'Mercurial {version} Inno Setup installer - x86 Windows (Python 3) '
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	111 '- does not require admin rights'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	112 )
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	113 EXE_PYTHON3_X64_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	114 'Mercurial {version} Inno Setup installer - x64 Windows (Python 3) '
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	115 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	116 )
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	117 MSI_PYTHON3_X86_DESCRIPTION = (
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	118 'Mercurial {version} MSI installer - x86 Windows (Python 3) '
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	119 '- requires admin rights'
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	120 )
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	121 MSI_PYTHON3_X64_DESCRIPTION = (
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	122 'Mercurial {version} MSI installer - x64 Windows (Python 3) '
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	123 '- requires admin rights'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	124 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	125
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	126
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	127 def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	128 commands = [
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	129 '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	130 'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	131 r'icacls %s /remove:g "NT Service\sshd"' % path,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	132 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	134 run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138 """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140 winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145 ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	147 ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	148
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	149 # Generate SSH key to use for communication.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	150 subprocess.run(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	151 [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	152 'ssh-keygen',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	153 '-t',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	154 'rsa',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	155 '-b',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	156 '4096',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	157 '-N',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	158 '',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	159 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	160 str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	161 ],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	162 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	163 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	164 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167 # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	169 winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170 winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171 fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178 fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 fh.write(' User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180 fh.write(' StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	181 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	182 fh.write(' IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	183
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	184 if not (hg_repo / '.hg').is_dir():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	185 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	186 '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	187 'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	188 )
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	189
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190 env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191 env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192 env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	195
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	196 res = subprocess.run(
48934 d953a42b157d automation: run hg with python3 Gregory Szorc <gregory.szorc@gmail.com> parents: 48405 diff changeset	197 ['python3', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	198 cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	199 env=env,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	200 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	201 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	202 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	203
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	204 full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	205
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	206 args = [
48934 d953a42b157d automation: run hg with python3 Gregory Szorc <gregory.szorc@gmail.com> parents: 48405 diff changeset	207 'python3',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	208 hg_bin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	209 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	210 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	211 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	212 'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42658 9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	213 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	214 # calculation works.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	215 'push',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	216 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	217 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	218 full_revision,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	219 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	220 'file(.hgtags)',
42309 5c242c427897 automation: do a force push to synchronize Gregory Szorc <gregory.szorc@gmail.com> parents: 42308 diff changeset	221 'ssh://%s/c:/hgdev/src' % public_ip,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223
42651 24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	224 res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	225
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	226 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	227 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	228 res.check_returncode()
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	230 run_powershell(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	231 winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	232 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237 def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238 """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239 run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	240
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	242 def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	243 """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	244
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	245 res = winrm_client.execute_ps(
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	246 r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	247 '\| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248 '\| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	249 '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250 )
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	251 return res[0]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254 def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255 """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258 matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259 local machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	260 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	263 dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264 print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	265 winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	266
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	267
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	268 def build_inno_installer(
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	269 winrm_client,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	270 arch: str,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	271 dest_path: pathlib.Path,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	272 version=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	273 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	276 Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	277 a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	278 """
48938 4561ec90d3c1 automation: delete code related to Python 2.7 support Gregory Szorc <gregory.szorc@gmail.com> parents: 48937 diff changeset	279 print('building Inno Setup installer for %s' % arch)
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	280
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	281 # TODO fix this limitation in packaging code
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	282 if not version:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	283 raise Exception("version string is required when building for Python 3")
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	284
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	285 if arch == "x86":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	286 target_triple = "i686-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	287 elif arch == "x64":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	288 target_triple = "x86_64-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	289 else:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	290 raise Exception("unhandled arch: %s" % arch)
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	291
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	292 ps = BUILD_INNO_PYTHON3.format(
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	293 pyoxidizer_target=target_triple,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	294 version=version,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	295 )
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	296
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	298 copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	300
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	301 def build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	302 winrm_client, python_version: str, arch: str, dest_path: pathlib.Path
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	303 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304 """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306 Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307 for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 """
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	309 print('Building Windows wheel for Python %s %s' % (python_version, arch))
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	310
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	311 ps = BUILD_WHEEL.format(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	312 python_version=python_version.replace(".", ""), arch=arch
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	313 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	314
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	315 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	316 copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	317
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	319 def build_wix_installer(
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	320 winrm_client,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	321 arch: str,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	322 dest_path: pathlib.Path,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	323 version=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	324 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	325 """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	326
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327 Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	328 """
48938 4561ec90d3c1 automation: delete code related to Python 2.7 support Gregory Szorc <gregory.szorc@gmail.com> parents: 48937 diff changeset	329 print('Building WiX installer for %s' % arch)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	330
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	331 # TODO fix this limitation in packaging code
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	332 if not version:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	333 raise Exception("version string is required when building for Python 3")
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	335 if arch == "x86":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	336 target_triple = "i686-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	337 elif arch == "x64":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	338 target_triple = "x86_64-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	339 else:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	340 raise Exception("unhandled arch: %s" % arch)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	341
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	342 ps = BUILD_WIX_PYTHON3.format(
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	343 pyoxidizer_target=target_triple,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	344 version=version,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	345 )
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	346
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351 def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352 """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355 ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	356 ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	357 ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	358 """
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	359 if not re.match(r'\d\.\d', python_version):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	360 raise ValueError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	361 r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	362 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	363
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364 if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365 raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	366
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	367 python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	368
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	369 ps = RUN_TESTS.format(
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	370 python_path=python_path,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	371 test_flags=test_flags or '',
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	372 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	373
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	374 run_powershell(winrm_client, ps)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	375
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	376
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	377 def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	378 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	379 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	380 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	381 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	382 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	383 dist_path / WHEEL_FILENAME_PYTHON39_X86.format(version=version),
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	384 dist_path / WHEEL_FILENAME_PYTHON39_X64.format(version=version),
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	385 dist_path / WHEEL_FILENAME_PYTHON310_X86.format(version=version),
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	386 dist_path / WHEEL_FILENAME_PYTHON310_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	387 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	388
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	389
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	390 def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	391 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	392 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	393 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	394 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	395 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	396 dist_path / WHEEL_FILENAME_PYTHON39_X86.format(version=version),
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	397 dist_path / WHEEL_FILENAME_PYTHON39_X64.format(version=version),
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	398 dist_path / WHEEL_FILENAME_PYTHON310_X86.format(version=version),
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	399 dist_path / WHEEL_FILENAME_PYTHON310_X64.format(version=version),
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	400 dist_path / EXE_FILENAME_PYTHON3_X86.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	401 dist_path / EXE_FILENAME_PYTHON3_X64.format(version=version),
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	402 dist_path / MSI_FILENAME_PYTHON3_X86.format(version=version),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	403 dist_path / MSI_FILENAME_PYTHON3_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	404 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	405
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	406
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	407 def generate_latest_dat(version: str):
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	408 python3_x86_exe_filename = EXE_FILENAME_PYTHON3_X86.format(version=version)
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	409 python3_x64_exe_filename = EXE_FILENAME_PYTHON3_X64.format(version=version)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	410 python3_x86_msi_filename = MSI_FILENAME_PYTHON3_X86.format(version=version)
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	411 python3_x64_msi_filename = MSI_FILENAME_PYTHON3_X64.format(version=version)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	412
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	413 entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	414 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	415 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	416 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	417 X86_USER_AGENT_PATTERN,
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	418 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	419 EXE_PYTHON3_X86_DESCRIPTION.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	420 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	421 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	422 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	423 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	424 X64_USER_AGENT_PATTERN,
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	425 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	426 EXE_PYTHON3_X64_DESCRIPTION.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	427 ),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	428 (
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	429 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	430 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	431 X86_USER_AGENT_PATTERN,
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	432 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_msi_filename),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	433 MSI_PYTHON3_X86_DESCRIPTION.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	434 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	435 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	436 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	437 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	438 X64_USER_AGENT_PATTERN,
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	439 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_msi_filename),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	440 MSI_PYTHON3_X64_DESCRIPTION.format(version=version),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	441 ),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	442 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	443
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	444 lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	445
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	446 return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	447
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	448
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	449 def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	450 """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	451
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	452 wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	453
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	454 for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	455 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	456 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	457
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	458 print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	459 pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	460
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	461
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	462 def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	463 dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	464 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	465 """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	466 all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	467
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	468 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	469 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	470 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	471
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	472 client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	473 client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	474 # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	475 print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	476 try:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	477 client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	478 except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	479 print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	480 raise
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	481
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	482 print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	483
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	484 print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	485 sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	486 print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	487
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	488 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	489 dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	490 print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	491
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	492 with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	493 data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	494
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	495 with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	496 fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	497 fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	498
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	499 latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	500
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	501 now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	502 backup_path = dist_path / (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	503 'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	504 )
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	505 print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	506
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	507 with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	508 latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	509
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	510 with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	511 fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	512
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	513 print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	514 latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	515 print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	516
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	517 with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	518 fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	519
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	520
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	521 def publish_artifacts(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	522 dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	523 version: str,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	524 pypi=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	525 mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	526 ssh_username=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	527 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	528 """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	529
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	530 Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	531 `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	532
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	533 `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	534 `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	535 """
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	536 if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	537 publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	538
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	539 if mercurial_scm_org:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	540 publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	541 dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	542 )

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/windows.py @ 49404:e4e33b779fa2 stable