mercurial-scm/hg-stable: contrib/automation/hgautomation/windows.py annotate

annotate contrib/automation/hgautomation/windows.py @ 48937:d7e064d509a0

automation: drop support for Python 2.7 in Windows environment We stop installing Python 2.7 in the Windows environment. We remove support for building Python 2.7 wheels and installers. There is still some Python 2.7 support cleanup to perform in automation. But this removes the biggest remaining chunk of references to 2.7. Differential Revision: https://phab.mercurial-scm.org/D12263

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Sat, 19 Feb 2022 18:42:31 -0700
parents	d953a42b157d
children	4561ec90d3c1

rev	line source
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	10 import datetime
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 import os
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	12 import paramiko
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 import pathlib
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import re
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import subprocess
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 import tempfile
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	18 from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	19 from .winrm import run_powershell
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 }
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29 Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	43 hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	47 BUILD_INNO_PYTHON3 = r'''
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	48 $Env:RUSTUP_HOME = "C:\hgdev\rustup"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	49 $Env:CARGO_HOME = "C:\hgdev\cargo"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	50 Set-Location C:\hgdev\src
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	51 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --pyoxidizer-target {pyoxidizer_target} --version {version}
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	52 if ($LASTEXITCODE -ne 0) {{
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	53 throw "process exited non-0: $LASTEXITCODE"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	54 }}
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	55 '''
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	56
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59 Set-Location C:\hgdev\src
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	60 C:\hgdev\python{python_version}-{arch}\python.exe -m pip wheel --wheel-dir dist .
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	62 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	66 BUILD_WIX_PYTHON3 = r'''
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	67 $Env:RUSTUP_HOME = "C:\hgdev\rustup"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	68 $Env:CARGO_HOME = "C:\hgdev\cargo"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	69 Set-Location C:\hgdev\src
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	70 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py wix --pyoxidizer-target {pyoxidizer_target} --version {version}
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	71 if ($LASTEXITCODE -ne 0) {{
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	72 throw "process exited non-0: $LASTEXITCODE"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	73 }}
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	74 '''
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	75
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	84
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	85 WHEEL_FILENAME_PYTHON37_X86 = 'mercurial-{version}-cp37-cp37m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	86 WHEEL_FILENAME_PYTHON37_X64 = 'mercurial-{version}-cp37-cp37m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	87 WHEEL_FILENAME_PYTHON38_X86 = 'mercurial-{version}-cp38-cp38-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	88 WHEEL_FILENAME_PYTHON38_X64 = 'mercurial-{version}-cp38-cp38-win_amd64.whl'
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	89 WHEEL_FILENAME_PYTHON39_X86 = 'mercurial-{version}-cp39-cp39-win32.whl'
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	90 WHEEL_FILENAME_PYTHON39_X64 = 'mercurial-{version}-cp39-cp39-win_amd64.whl'
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	91 WHEEL_FILENAME_PYTHON310_X86 = 'mercurial-{version}-cp310-cp310-win32.whl'
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	92 WHEEL_FILENAME_PYTHON310_X64 = 'mercurial-{version}-cp310-cp310-win_amd64.whl'
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	93
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	94 EXE_FILENAME_PYTHON3_X86 = 'Mercurial-{version}-x86.exe'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	95 EXE_FILENAME_PYTHON3_X64 = 'Mercurial-{version}-x64.exe'
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	96
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	97 MSI_FILENAME_PYTHON3_X86 = 'mercurial-{version}-x86.msi'
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	98 MSI_FILENAME_PYTHON3_X64 = 'mercurial-{version}-x64.msi'
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	99
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	100 MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	101
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	102 X86_USER_AGENT_PATTERN = '.Windows.'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	103 X64_USER_AGENT_PATTERN = '.Windows.(WOW\|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	104
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	105 # TODO remove Python version once Python 2 is dropped.
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	106 EXE_PYTHON3_X86_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	107 'Mercurial {version} Inno Setup installer - x86 Windows (Python 3) '
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	108 '- does not require admin rights'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	109 )
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	110 EXE_PYTHON3_X64_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	111 'Mercurial {version} Inno Setup installer - x64 Windows (Python 3) '
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	112 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	113 )
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	114 MSI_PYTHON3_X86_DESCRIPTION = (
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	115 'Mercurial {version} MSI installer - x86 Windows (Python 3) '
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	116 '- requires admin rights'
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	117 )
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	118 MSI_PYTHON3_X64_DESCRIPTION = (
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	119 'Mercurial {version} MSI installer - x64 Windows (Python 3) '
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	120 '- requires admin rights'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	121 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	122
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	124 def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	125 commands = [
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	126 '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	127 'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	128 r'icacls %s /remove:g "NT Service\sshd"' % path,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	129 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	130
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	131 run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	132
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	134 def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135 """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139 with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140 temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144 ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 # Generate SSH key to use for communication.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	147 subprocess.run(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	148 [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	149 'ssh-keygen',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	150 '-t',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	151 'rsa',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	152 '-b',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	153 '4096',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	154 '-N',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	155 '',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	156 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	157 str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	158 ],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	159 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	160 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	161 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163 # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	164 # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165 authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167 winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	169
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170 public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172 ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176 fh.write(' User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 fh.write(' StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 fh.write(' IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	181 if not (hg_repo / '.hg').is_dir():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	182 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	183 '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	184 'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	185 )
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	186
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189 env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191 hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193 res = subprocess.run(
48934 d953a42b157d automation: run hg with python3 Gregory Szorc <gregory.szorc@gmail.com> parents: 48405 diff changeset	194 ['python3', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	195 cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	196 env=env,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	197 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	198 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	199 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	201 full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	202
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	203 args = [
48934 d953a42b157d automation: run hg with python3 Gregory Szorc <gregory.szorc@gmail.com> parents: 48405 diff changeset	204 'python3',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	205 hg_bin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	206 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	207 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	208 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	209 'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42658 9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	210 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	211 # calculation works.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	212 'push',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	213 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	214 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	215 full_revision,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	216 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	217 'file(.hgtags)',
42309 5c242c427897 automation: do a force push to synchronize Gregory Szorc <gregory.szorc@gmail.com> parents: 42308 diff changeset	218 'ssh://%s/c:/hgdev/src' % public_ip,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220
42651 24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	221 res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	222
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	223 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	224 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	225 res.check_returncode()
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	227 run_powershell(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	228 winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	229 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	230
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231 # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	232
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235 """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239 def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	240 """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	242 res = winrm_client.execute_ps(
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	243 r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	244 '\| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	245 '\| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	246 '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	247 )
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248 return res[0]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	249
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	251 def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252 """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254 Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255 matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256 local machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258 latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259 source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	260 dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	263
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	265 def build_inno_installer(
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	266 winrm_client,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	267 python_version: int,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	268 arch: str,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	269 dest_path: pathlib.Path,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	270 version=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	271 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	272 """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	273
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275 a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	276 """
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	277 print(
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	278 'building Inno Setup installer for Python %d %s'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	279 % (python_version, arch)
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	280 )
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	281
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	282 # TODO fix this limitation in packaging code
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	283 if not version:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	284 raise Exception("version string is required when building for Python 3")
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	285
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	286 if arch == "x86":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	287 target_triple = "i686-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	288 elif arch == "x64":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	289 target_triple = "x86_64-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	290 else:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	291 raise Exception("unhandled arch: %s" % arch)
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	293 ps = BUILD_INNO_PYTHON3.format(
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	294 pyoxidizer_target=target_triple,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	295 version=version,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	296 )
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	297
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	298 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299 copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	300
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	302 def build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	303 winrm_client, python_version: str, arch: str, dest_path: pathlib.Path
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	304 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305 """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307 Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	309 """
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	310 print('Building Windows wheel for Python %s %s' % (python_version, arch))
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	311
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	312 ps = BUILD_WHEEL.format(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	313 python_version=python_version.replace(".", ""), arch=arch
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	314 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	315
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	316 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	317 copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	319
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	320 def build_wix_installer(
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	321 winrm_client,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	322 python_version: int,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	323 arch: str,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	324 dest_path: pathlib.Path,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	325 version=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	326 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327 """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	328
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329 Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	330 """
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	331 print('Building WiX installer for Python %d %s' % (python_version, arch))
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	332
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	333 # TODO fix this limitation in packaging code
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	334 if not version:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	335 raise Exception("version string is required when building for Python 3")
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	337 if arch == "x86":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	338 target_triple = "i686-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	339 elif arch == "x64":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	340 target_triple = "x86_64-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	341 else:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	342 raise Exception("unhandled arch: %s" % arch)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	343
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	344 ps = BUILD_WIX_PYTHON3.format(
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	345 pyoxidizer_target=target_triple,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	346 version=version,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	347 )
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	348
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353 def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	356 ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	357 ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	358 ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	359 ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	360 """
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	361 if not re.match(r'\d\.\d', python_version):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	362 raise ValueError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	363 r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	364 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	366 if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	367 raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	368
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	369 python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	370
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	371 ps = RUN_TESTS.format(
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	372 python_path=python_path,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	373 test_flags=test_flags or '',
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	374 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	375
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	376 run_powershell(winrm_client, ps)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	377
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	378
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	379 def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	380 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	381 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	382 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	383 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	384 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	385 dist_path / WHEEL_FILENAME_PYTHON39_X86.format(version=version),
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	386 dist_path / WHEEL_FILENAME_PYTHON39_X64.format(version=version),
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	387 dist_path / WHEEL_FILENAME_PYTHON310_X86.format(version=version),
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	388 dist_path / WHEEL_FILENAME_PYTHON310_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	389 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	390
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	391
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	392 def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	393 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	394 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	395 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	396 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	397 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	398 dist_path / WHEEL_FILENAME_PYTHON39_X86.format(version=version),
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	399 dist_path / WHEEL_FILENAME_PYTHON39_X64.format(version=version),
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	400 dist_path / WHEEL_FILENAME_PYTHON310_X86.format(version=version),
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	401 dist_path / WHEEL_FILENAME_PYTHON310_X64.format(version=version),
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	402 dist_path / EXE_FILENAME_PYTHON3_X86.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	403 dist_path / EXE_FILENAME_PYTHON3_X64.format(version=version),
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	404 dist_path / MSI_FILENAME_PYTHON3_X86.format(version=version),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	405 dist_path / MSI_FILENAME_PYTHON3_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	406 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	407
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	408
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	409 def generate_latest_dat(version: str):
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	410 python3_x86_exe_filename = EXE_FILENAME_PYTHON3_X86.format(version=version)
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	411 python3_x64_exe_filename = EXE_FILENAME_PYTHON3_X64.format(version=version)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	412 python3_x86_msi_filename = MSI_FILENAME_PYTHON3_X86.format(version=version)
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	413 python3_x64_msi_filename = MSI_FILENAME_PYTHON3_X64.format(version=version)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	414
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	415 entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	416 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	417 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	418 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	419 X86_USER_AGENT_PATTERN,
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	420 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	421 EXE_PYTHON3_X86_DESCRIPTION.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	422 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	423 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	424 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	425 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	426 X64_USER_AGENT_PATTERN,
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	427 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	428 EXE_PYTHON3_X64_DESCRIPTION.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	429 ),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	430 (
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	431 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	432 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	433 X86_USER_AGENT_PATTERN,
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	434 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_msi_filename),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	435 MSI_PYTHON3_X86_DESCRIPTION.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	436 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	437 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	438 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	439 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	440 X64_USER_AGENT_PATTERN,
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	441 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_msi_filename),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	442 MSI_PYTHON3_X64_DESCRIPTION.format(version=version),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	443 ),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	444 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	445
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	446 lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	447
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	448 return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	449
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	450
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	451 def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	452 """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	453
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	454 wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	455
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	456 for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	457 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	458 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	459
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	460 print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	461 pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	462
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	463
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	464 def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	465 dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	466 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	467 """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	468 all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	469
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	470 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	471 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	472 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	473
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	474 client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	475 client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	476 # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	477 print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	478 try:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	479 client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	480 except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	481 print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	482 raise
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	483
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	484 print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	485
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	486 print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	487 sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	488 print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	489
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	490 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	491 dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	492 print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	493
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	494 with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	495 data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	496
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	497 with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	498 fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	499 fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	500
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	501 latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	502
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	503 now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	504 backup_path = dist_path / (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	505 'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	506 )
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	507 print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	508
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	509 with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	510 latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	511
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	512 with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	513 fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	514
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	515 print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	516 latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	517 print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	518
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	519 with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	520 fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	521
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	522
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	523 def publish_artifacts(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	524 dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	525 version: str,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	526 pypi=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	527 mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	528 ssh_username=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	529 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	530 """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	531
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	532 Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	533 `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	534
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	535 `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	536 `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	537 """
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	538 if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	539 publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	540
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	541 if mercurial_scm_org:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	542 publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	543 dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	544 )

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/windows.py @ 48937:d7e064d509a0