mercurial-scm/hg-stable: contrib/automation/hgautomation/windows.py annotate

annotate contrib/automation/hgautomation/windows.py @ 48938:4561ec90d3c1

automation: delete code related to Python 2.7 support The building of Inno and WiX installers took a python_version argument that allowed us to specify "2" or "3" for the major Python version. Since we no longer support Python 2, we can delete this argument and everything feeding into it. Differential Revision: https://phab.mercurial-scm.org/D12264

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Sat, 19 Feb 2022 18:42:12 -0700
parents	d7e064d509a0
children	e4e33b779fa2

rev	line source
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	10 import datetime
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 import os
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	12 import paramiko
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 import pathlib
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import re
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import subprocess
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 import tempfile
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	18 from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	19 from .winrm import run_powershell
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 }
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29 Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	43 hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	47 BUILD_INNO_PYTHON3 = r'''
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	48 $Env:RUSTUP_HOME = "C:\hgdev\rustup"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	49 $Env:CARGO_HOME = "C:\hgdev\cargo"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	50 Set-Location C:\hgdev\src
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	51 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --pyoxidizer-target {pyoxidizer_target} --version {version}
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	52 if ($LASTEXITCODE -ne 0) {{
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	53 throw "process exited non-0: $LASTEXITCODE"
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	54 }}
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	55 '''
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	56
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59 Set-Location C:\hgdev\src
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	60 C:\hgdev\python{python_version}-{arch}\python.exe -m pip wheel --wheel-dir dist .
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	62 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	66 BUILD_WIX_PYTHON3 = r'''
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	67 $Env:RUSTUP_HOME = "C:\hgdev\rustup"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	68 $Env:CARGO_HOME = "C:\hgdev\cargo"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	69 Set-Location C:\hgdev\src
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	70 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py wix --pyoxidizer-target {pyoxidizer_target} --version {version}
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	71 if ($LASTEXITCODE -ne 0) {{
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	72 throw "process exited non-0: $LASTEXITCODE"
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	73 }}
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	74 '''
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	75
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	84
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	85 WHEEL_FILENAME_PYTHON37_X86 = 'mercurial-{version}-cp37-cp37m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	86 WHEEL_FILENAME_PYTHON37_X64 = 'mercurial-{version}-cp37-cp37m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	87 WHEEL_FILENAME_PYTHON38_X86 = 'mercurial-{version}-cp38-cp38-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	88 WHEEL_FILENAME_PYTHON38_X64 = 'mercurial-{version}-cp38-cp38-win_amd64.whl'
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	89 WHEEL_FILENAME_PYTHON39_X86 = 'mercurial-{version}-cp39-cp39-win32.whl'
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	90 WHEEL_FILENAME_PYTHON39_X64 = 'mercurial-{version}-cp39-cp39-win_amd64.whl'
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	91 WHEEL_FILENAME_PYTHON310_X86 = 'mercurial-{version}-cp310-cp310-win32.whl'
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	92 WHEEL_FILENAME_PYTHON310_X64 = 'mercurial-{version}-cp310-cp310-win_amd64.whl'
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	93
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	94 EXE_FILENAME_PYTHON3_X86 = 'Mercurial-{version}-x86.exe'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	95 EXE_FILENAME_PYTHON3_X64 = 'Mercurial-{version}-x64.exe'
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	96
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	97 MSI_FILENAME_PYTHON3_X86 = 'mercurial-{version}-x86.msi'
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	98 MSI_FILENAME_PYTHON3_X64 = 'mercurial-{version}-x64.msi'
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	99
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	100 MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	101
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	102 X86_USER_AGENT_PATTERN = '.Windows.'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	103 X64_USER_AGENT_PATTERN = '.Windows.(WOW\|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	104
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	105 # TODO remove Python version once Python 2 is dropped.
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	106 EXE_PYTHON3_X86_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	107 'Mercurial {version} Inno Setup installer - x86 Windows (Python 3) '
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	108 '- does not require admin rights'
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	109 )
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	110 EXE_PYTHON3_X64_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	111 'Mercurial {version} Inno Setup installer - x64 Windows (Python 3) '
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	112 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	113 )
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	114 MSI_PYTHON3_X86_DESCRIPTION = (
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	115 'Mercurial {version} MSI installer - x86 Windows (Python 3) '
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	116 '- requires admin rights'
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	117 )
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	118 MSI_PYTHON3_X64_DESCRIPTION = (
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	119 'Mercurial {version} MSI installer - x64 Windows (Python 3) '
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	120 '- requires admin rights'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	121 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	122
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	124 def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	125 commands = [
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	126 '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	127 'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	128 r'icacls %s /remove:g "NT Service\sshd"' % path,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	129 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	130
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	131 run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	132
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	134 def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135 """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139 with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140 temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144 ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 # Generate SSH key to use for communication.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	147 subprocess.run(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	148 [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	149 'ssh-keygen',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	150 '-t',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	151 'rsa',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	152 '-b',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	153 '4096',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	154 '-N',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	155 '',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	156 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	157 str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	158 ],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	159 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	160 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	161 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163 # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	164 # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165 authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167 winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	169
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170 public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172 ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176 fh.write(' User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 fh.write(' StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 fh.write(' IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	181 if not (hg_repo / '.hg').is_dir():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	182 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	183 '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	184 'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	185 )
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	186
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189 env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191 hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193 res = subprocess.run(
48934 d953a42b157d automation: run hg with python3 Gregory Szorc <gregory.szorc@gmail.com> parents: 48405 diff changeset	194 ['python3', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	195 cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	196 env=env,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	197 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	198 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	199 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	201 full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	202
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	203 args = [
48934 d953a42b157d automation: run hg with python3 Gregory Szorc <gregory.szorc@gmail.com> parents: 48405 diff changeset	204 'python3',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	205 hg_bin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	206 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	207 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	208 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	209 'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42658 9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	210 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	211 # calculation works.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	212 'push',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	213 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	214 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	215 full_revision,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	216 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	217 'file(.hgtags)',
42309 5c242c427897 automation: do a force push to synchronize Gregory Szorc <gregory.szorc@gmail.com> parents: 42308 diff changeset	218 'ssh://%s/c:/hgdev/src' % public_ip,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220
42651 24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	221 res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	222
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	223 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	224 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	225 res.check_returncode()
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	227 run_powershell(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	228 winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	229 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	230
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231 # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	232
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235 """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239 def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	240 """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	242 res = winrm_client.execute_ps(
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	243 r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	244 '\| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	245 '\| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	246 '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	247 )
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248 return res[0]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	249
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	251 def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252 """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254 Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255 matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256 local machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258 latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259 source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	260 dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	263
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	265 def build_inno_installer(
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	266 winrm_client,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	267 arch: str,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	268 dest_path: pathlib.Path,
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	269 version=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	270 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	271 """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	272
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	273 Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275 """
48938 4561ec90d3c1 automation: delete code related to Python 2.7 support Gregory Szorc <gregory.szorc@gmail.com> parents: 48937 diff changeset	276 print('building Inno Setup installer for %s' % arch)
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	277
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	278 # TODO fix this limitation in packaging code
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	279 if not version:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	280 raise Exception("version string is required when building for Python 3")
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	281
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	282 if arch == "x86":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	283 target_triple = "i686-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	284 elif arch == "x64":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	285 target_triple = "x86_64-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	286 else:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	287 raise Exception("unhandled arch: %s" % arch)
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	288
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	289 ps = BUILD_INNO_PYTHON3.format(
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	290 pyoxidizer_target=target_triple,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	291 version=version,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	292 )
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	293
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295 copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	296
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	298 def build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	299 winrm_client, python_version: str, arch: str, dest_path: pathlib.Path
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	300 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301 """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	302
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	303 Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304 for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305 """
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	306 print('Building Windows wheel for Python %s %s' % (python_version, arch))
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	307
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	308 ps = BUILD_WHEEL.format(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	309 python_version=python_version.replace(".", ""), arch=arch
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	310 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	311
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313 copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	315
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	316 def build_wix_installer(
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	317 winrm_client,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	318 arch: str,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	319 dest_path: pathlib.Path,
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	320 version=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	321 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	322 """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	324 Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	325 """
48938 4561ec90d3c1 automation: delete code related to Python 2.7 support Gregory Szorc <gregory.szorc@gmail.com> parents: 48937 diff changeset	326 print('Building WiX installer for %s' % arch)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	327
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	328 # TODO fix this limitation in packaging code
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	329 if not version:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	330 raise Exception("version string is required when building for Python 3")
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	331
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	332 if arch == "x86":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	333 target_triple = "i686-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	334 elif arch == "x64":
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	335 target_triple = "x86_64-pc-windows-msvc"
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	336 else:
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	337 raise Exception("unhandled arch: %s" % arch)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	338
48937 d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	339 ps = BUILD_WIX_PYTHON3.format(
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	340 pyoxidizer_target=target_triple,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	341 version=version,
d7e064d509a0 automation: drop support for Python 2.7 in Windows environment Gregory Szorc <gregory.szorc@gmail.com> parents: 48934 diff changeset	342 )
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	343
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	344 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	345 copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	346
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351 ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352 ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353 ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355 """
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	356 if not re.match(r'\d\.\d', python_version):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	357 raise ValueError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	358 r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	359 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	360
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	361 if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	362 raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	363
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364 python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	366 ps = RUN_TESTS.format(
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	367 python_path=python_path,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	368 test_flags=test_flags or '',
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45753 diff changeset	369 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	370
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	371 run_powershell(winrm_client, ps)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	372
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	373
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	374 def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	375 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	376 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	377 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	378 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	379 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	380 dist_path / WHEEL_FILENAME_PYTHON39_X86.format(version=version),
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	381 dist_path / WHEEL_FILENAME_PYTHON39_X64.format(version=version),
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	382 dist_path / WHEEL_FILENAME_PYTHON310_X86.format(version=version),
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	383 dist_path / WHEEL_FILENAME_PYTHON310_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	384 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	385
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	386
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	387 def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	388 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	389 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	390 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	391 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	392 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
45753 d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	393 dist_path / WHEEL_FILENAME_PYTHON39_X86.format(version=version),
d1ce0ffdd3ce automation: upload Python 3.9 Windows wheels Gregory Szorc <gregory.szorc@gmail.com> parents: 44757 diff changeset	394 dist_path / WHEEL_FILENAME_PYTHON39_X64.format(version=version),
48405 fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	395 dist_path / WHEEL_FILENAME_PYTHON310_X86.format(version=version),
fc1ba19ec4a0 automation: support Python 3.10 on Windows Gregory Szorc <gregory.szorc@gmail.com> parents: 45957 diff changeset	396 dist_path / WHEEL_FILENAME_PYTHON310_X64.format(version=version),
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	397 dist_path / EXE_FILENAME_PYTHON3_X86.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	398 dist_path / EXE_FILENAME_PYTHON3_X64.format(version=version),
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	399 dist_path / MSI_FILENAME_PYTHON3_X86.format(version=version),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	400 dist_path / MSI_FILENAME_PYTHON3_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	401 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	402
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	403
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	404 def generate_latest_dat(version: str):
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	405 python3_x86_exe_filename = EXE_FILENAME_PYTHON3_X86.format(version=version)
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	406 python3_x64_exe_filename = EXE_FILENAME_PYTHON3_X64.format(version=version)
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	407 python3_x86_msi_filename = MSI_FILENAME_PYTHON3_X86.format(version=version)
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	408 python3_x64_msi_filename = MSI_FILENAME_PYTHON3_X64.format(version=version)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	409
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	410 entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	411 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	412 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	413 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	414 X86_USER_AGENT_PATTERN,
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	415 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	416 EXE_PYTHON3_X86_DESCRIPTION.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	417 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	418 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	419 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	420 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	421 X64_USER_AGENT_PATTERN,
44756 802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	422 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	423 EXE_PYTHON3_X64_DESCRIPTION.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	424 ),
802ee93c205d automation: support building Python 3 Inno installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44755 diff changeset	425 (
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	426 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	427 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	428 X86_USER_AGENT_PATTERN,
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	429 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_msi_filename),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	430 MSI_PYTHON3_X86_DESCRIPTION.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	431 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	432 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	433 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	434 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	435 X64_USER_AGENT_PATTERN,
44757 5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	436 '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_msi_filename),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	437 MSI_PYTHON3_X64_DESCRIPTION.format(version=version),
5e788dc7fb5d automation: support building Python 3 MSI installers Gregory Szorc <gregory.szorc@gmail.com> parents: 44756 diff changeset	438 ),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	439 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	440
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	441 lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	442
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	443 return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	444
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	445
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	446 def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	447 """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	448
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	449 wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	450
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	451 for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	452 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	453 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	454
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	455 print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	456 pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	457
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	458
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	459 def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	460 dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	461 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	462 """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	463 all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	464
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	465 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	466 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	467 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	468
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	469 client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	470 client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	471 # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	472 print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	473 try:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	474 client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	475 except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	476 print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	477 raise
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	478
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	479 print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	480
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	481 print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	482 sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	483 print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	484
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	485 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	486 dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	487 print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	488
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	489 with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	490 data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	491
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	492 with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	493 fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	494 fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	495
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	496 latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	497
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	498 now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	499 backup_path = dist_path / (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	500 'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	501 )
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	502 print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	503
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	504 with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	505 latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	506
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	507 with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	508 fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	509
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	510 print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	511 latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	512 print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	513
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	514 with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	515 fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	516
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	517
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	518 def publish_artifacts(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	519 dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	520 version: str,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	521 pypi=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	522 mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	523 ssh_username=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	524 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	525 """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	526
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	527 Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	528 `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	529
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	530 `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	531 `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	532 """
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	533 if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	534 publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	535
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	536 if mercurial_scm_org:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	537 publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	538 dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	539 )

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/windows.py @ 48938:4561ec90d3c1