mercurial-scm/hg-stable: contrib/automation/hgautomation/windows.py annotate

annotate contrib/automation/hgautomation/windows.py @ 44754:9ade217b550d stable

packaging: add -python2 to Windows installer filenames We just taught the Windows installers to produce Python 3 variants built with PyOxidizer. Our plan is to publish both Python 2 and Python 3 versions of the installers for Mercurial 5.4. This commit teaches the Inno and WiX installers to add an optional string suffix to the installer name. On Python 2, that suffix is "-python2." We reserve the existing name for the Python 3 installers, which we want to make the default. Differential Revision: https://phab.mercurial-scm.org/D8479

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Thu, 23 Apr 2020 18:48:36 -0700
parents	9d441f820c8b
children	47609da15379

rev	line source
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	10 import datetime
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 import os
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	12 import paramiko
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 import pathlib
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import re
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import subprocess
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 import tempfile
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	18 from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	19 from .winrm import run_powershell
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 # PowerShell commands to activate a Visual Studio 2008 environment.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 # This is essentially a port of vcvarsall.bat to PowerShell.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 ACTIVATE_VC9_AMD64 = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 Write-Output "activating Visual Studio 2008 environment for AMD64"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 $root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 $Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 $Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29 $Env:PATH = "${root}\VC\Bin\amd64;${root}\WinSDK\Bin\x64;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 $Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31 $Env:LIB = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIB"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 $Env:LIBPATH = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIBPATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 ACTIVATE_VC9_X86 = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 Write-Output "activating Visual Studio 2008 environment for x86"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 $root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 $Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 $Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 $Env:PATH = "${root}\VC\Bin;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 $Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:INCLUDE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 $Env:LIB = "${root}\VC\Lib;${root}\WinSDK\Lib;$Env:LIB"
42601 862f6bddacce automation: correct the path separator in LIBPATH on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 42309 diff changeset	43 $Env:LIBPATH = "${root}\VC\lib;${root}\WinSDK\Lib;$Env:LIBPATH"
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46 HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	47 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	48 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	49 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	50 if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 }
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	53 Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	56 HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	60 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	62 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63 hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	66 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	67 hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	68 Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	69 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	71 BUILD_INNO = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	72 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	73 $python = "C:\hgdev\python27-{arch}\python.exe"
43561 081a77df7bc6 packaging: consolidate CLI functionality into packaging.py Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	74 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --python $python
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 Set-Location C:\hgdev\src
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	82 C:\hgdev\python{python_version}-{arch}\python.exe -m pip wheel --wheel-dir dist .
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	86 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	87
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	88 BUILD_WIX = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	89 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	90 $python = "C:\hgdev\python27-{arch}\python.exe"
43561 081a77df7bc6 packaging: consolidate CLI functionality into packaging.py Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	91 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py wix --python $python {extra_args}
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	92 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	93 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	94 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	95 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	96
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	97 RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	98 C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	100 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	102 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	103
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	104 WHEEL_FILENAME_PYTHON27_X86 = 'mercurial-{version}-cp27-cp27m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	105 WHEEL_FILENAME_PYTHON27_X64 = 'mercurial-{version}-cp27-cp27m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	106 WHEEL_FILENAME_PYTHON37_X86 = 'mercurial-{version}-cp37-cp37m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	107 WHEEL_FILENAME_PYTHON37_X64 = 'mercurial-{version}-cp37-cp37m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	108 WHEEL_FILENAME_PYTHON38_X86 = 'mercurial-{version}-cp38-cp38-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	109 WHEEL_FILENAME_PYTHON38_X64 = 'mercurial-{version}-cp38-cp38-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	110
44754 9ade217b550d packaging: add -python2 to Windows installer filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 44753 diff changeset	111 X86_EXE_FILENAME = 'Mercurial-{version}-x86-python2.exe'
9ade217b550d packaging: add -python2 to Windows installer filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 44753 diff changeset	112 X64_EXE_FILENAME = 'Mercurial-{version}-x64-python2.exe'
9ade217b550d packaging: add -python2 to Windows installer filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 44753 diff changeset	113 X86_MSI_FILENAME = 'mercurial-{version}-x86-python2.msi'
9ade217b550d packaging: add -python2 to Windows installer filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 44753 diff changeset	114 X64_MSI_FILENAME = 'mercurial-{version}-x64-python2.msi'
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	115
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	116 MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	117
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	118 X86_USER_AGENT_PATTERN = '.Windows.'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	119 X64_USER_AGENT_PATTERN = '.Windows.(WOW\|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	120
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	121 X86_EXE_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	122 'Mercurial {version} Inno Setup installer - x86 Windows '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	123 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	124 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	125 X64_EXE_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	126 'Mercurial {version} Inno Setup installer - x64 Windows '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	127 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	128 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	129 X86_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	130 'Mercurial {version} MSI installer - x86 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	131 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	132 X64_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	133 'Mercurial {version} MSI installer - x64 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	134 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	135
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 def get_vc_prefix(arch):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138 if arch == 'x86':
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139 return ACTIVATE_VC9_X86
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140 elif arch == 'x64':
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141 return ACTIVATE_VC9_AMD64
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 else:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 raise ValueError('illegal arch: %s; must be x86 or x64' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	145
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	147 commands = [
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	148 '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	149 'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	150 r'icacls %s /remove:g "NT Service\sshd"' % path,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	151 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	152
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	153 run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	157 """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	158
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	159 winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	160
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	161 with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162 temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	164 ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165 ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	167
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	168 # Generate SSH key to use for communication.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	169 subprocess.run(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	170 [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	171 'ssh-keygen',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	172 '-t',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	173 'rsa',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	174 '-b',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	175 '4096',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	176 '-N',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	177 '',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	178 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	179 str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	180 ],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	181 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	182 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	183 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185 # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	186 # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189 winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190 fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192 public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	195
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	196 with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	197 fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	198 fh.write(' User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	199 fh.write(' StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	201 fh.write(' IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	202
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	203 if not (hg_repo / '.hg').is_dir():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	204 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	205 '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	206 'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	207 )
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	208
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	209 env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210 env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	212
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215 res = subprocess.run(
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 ['python2.7', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	217 cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	218 env=env,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	219 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	220 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	221 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223 full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	224
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	225 args = [
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	226 'python2.7',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	227 hg_bin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	228 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	229 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	230 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	231 'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42658 9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	232 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	233 # calculation works.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	234 'push',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	235 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	236 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	237 full_revision,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	238 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	239 'file(.hgtags)',
42309 5c242c427897 automation: do a force push to synchronize Gregory Szorc <gregory.szorc@gmail.com> parents: 42308 diff changeset	240 'ssh://%s/c:/hgdev/src' % public_ip,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	241 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	242
42651 24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	243 res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	244
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	245 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	246 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	247 res.check_returncode()
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	249 run_powershell(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	250 winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	251 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253 # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256 def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258 run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	260
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	263
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264 res = winrm_client.execute_ps(
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	265 r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	266 '\| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	267 '\| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	268 '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	269 )
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	270 return res[0]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	271
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	272
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	273 def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	276 Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	277 matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	278 local machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	279 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	280 latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	281 source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	282 dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	283 print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	284 winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	285
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	286
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	287 def build_inno_installer(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	288 winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	289 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	290 """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	291
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292 Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293 a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295 print('building Inno Setup installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	296
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297 extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	298 if version:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299 extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	300
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	301 ps = get_vc_prefix(arch) + BUILD_INNO.format(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	302 arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	303 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305 copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	308 def build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	309 winrm_client, python_version: str, arch: str, dest_path: pathlib.Path
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	310 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	311 """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313 Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314 for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	315 """
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	316 print('Building Windows wheel for Python %s %s' % (python_version, arch))
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	317
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	318 ps = BUILD_WHEEL.format(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	319 python_version=python_version.replace(".", ""), arch=arch
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	320 )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	321
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	322 # Python 2.7 requires an activated environment.
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	323 if python_version == "2.7":
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	324 ps = get_vc_prefix(arch) + ps
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	325
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	326 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327 copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	328
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	330 def build_wix_installer(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	331 winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	332 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333 """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 print('Building WiX installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338 extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 if version:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340 extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	341
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	342 ps = get_vc_prefix(arch) + BUILD_WIX.format(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	343 arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	344 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	345 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	346 copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352 ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353 ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355 ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	356 """
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	357 if not re.match(r'\d\.\d', python_version):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	358 raise ValueError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	359 r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	360 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	361
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	362 if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	363 raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365 python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	366
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	367 ps = RUN_TESTS.format(python_path=python_path, test_flags=test_flags or '',)
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	368
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	369 run_powershell(winrm_client, ps)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	370
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	371
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	372 def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	373 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	374 dist_path / WHEEL_FILENAME_PYTHON27_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	375 dist_path / WHEEL_FILENAME_PYTHON27_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	376 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	377 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	378 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	379 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	380 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	381
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	382
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	383 def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	384 return (
44753 9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	385 dist_path / WHEEL_FILENAME_PYTHON27_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	386 dist_path / WHEEL_FILENAME_PYTHON27_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	387 dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	388 dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	389 dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43561 diff changeset	390 dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	391 dist_path / X86_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	392 dist_path / X64_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	393 dist_path / X86_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	394 dist_path / X64_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	395 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	396
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	397
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	398 def generate_latest_dat(version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	399 x86_exe_filename = X86_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	400 x64_exe_filename = X64_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	401 x86_msi_filename = X86_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	402 x64_msi_filename = X64_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	403
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	404 entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	405 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	406 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	407 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	408 X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	409 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	410 X86_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	411 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	412 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	413 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	414 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	415 X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	416 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	417 X64_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	418 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	419 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	420 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	421 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	422 X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	423 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_msi_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	424 X86_MSI_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	425 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	426 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	427 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	428 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	429 X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	430 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_msi_filename),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	431 X64_MSI_DESCRIPTION.format(version=version),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	432 ),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	433 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	434
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	435 lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	436
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	437 return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	438
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	439
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	440 def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	441 """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	442
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	443 wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	444
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	445 for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	446 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	447 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	448
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	449 print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	450 pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	451
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	452
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	453 def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	454 dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	455 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	456 """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	457 all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	458
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	459 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	460 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	461 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	462
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	463 client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	464 client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	465 # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	466 print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	467 try:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	468 client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	469 except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	470 print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	471 raise
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	472
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	473 print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	474
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	475 print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	476 sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	477 print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	478
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	479 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	480 dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	481 print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	482
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	483 with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	484 data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	485
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	486 with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	487 fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	488 fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	489
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	490 latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	491
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	492 now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	493 backup_path = dist_path / (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	494 'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	495 )
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	496 print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	497
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	498 with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	499 latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	500
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	501 with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	502 fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	503
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	504 print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	505 latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	506 print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	507
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	508 with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	509 fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	510
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	511
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	512 def publish_artifacts(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	513 dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	514 version: str,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	515 pypi=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	516 mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	517 ssh_username=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	518 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	519 """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	520
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	521 Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	522 `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	523
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	524 `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	525 `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	526 """
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	527 if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	528 publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	529
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	530 if mercurial_scm_org:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	531 publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	532 dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	533 )

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/windows.py @ 44754:9ade217b550d stable