mercurial-scm/hg-stable: contrib/automation/hgautomation/windows.py annotate

annotate contrib/automation/hgautomation/windows.py @ 43561:081a77df7bc6

packaging: consolidate CLI functionality into packaging.py Consolidating functionality for invoking code in the hgpackaging package through a single CLI entry point will make things simpler when we add more complexity to that package. For example, it will allow us to run things out of a virtualenv with third party packages. This commit consolidates functionality from the Inno and WiX build.py scripts into a new packaging.py script. That script simply creates a virtualenv and runs the CLI functionality in it. The new virtualenv is populated with jinja2 because I felt it easier to incorporate requirements file processing in this commit and we will soon use jinja2 in an upcoming commit. The unified CLI functionality will also make it easier to script other packaging workflows going forward. e.g. RPM, Debian, and macOS packaging. Differential Revision: https://phab.mercurial-scm.org/D7156

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Wed, 23 Oct 2019 18:37:36 -0700
parents	2372284d9457
children	9d441f820c8b

rev	line source
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	10 import datetime
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11 import os
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	12 import paramiko
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13 import pathlib
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import re
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import subprocess
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16 import tempfile
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	17
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	18 from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	19 from .winrm import run_powershell
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	20
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 # PowerShell commands to activate a Visual Studio 2008 environment.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 # This is essentially a port of vcvarsall.bat to PowerShell.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 ACTIVATE_VC9_AMD64 = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 Write-Output "activating Visual Studio 2008 environment for AMD64"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 $root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 $Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28 $Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29 $Env:PATH = "${root}\VC\Bin\amd64;${root}\WinSDK\Bin\x64;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 $Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31 $Env:LIB = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIB"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 $Env:LIBPATH = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIBPATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 ACTIVATE_VC9_X86 = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 Write-Output "activating Visual Studio 2008 environment for x86"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37 $root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 $Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 $Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 $Env:PATH = "${root}\VC\Bin;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 $Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:INCLUDE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 $Env:LIB = "${root}\VC\Lib;${root}\WinSDK\Lib;$Env:LIB"
42601 862f6bddacce automation: correct the path separator in LIBPATH on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 42309 diff changeset	43 $Env:LIBPATH = "${root}\VC\lib;${root}\WinSDK\Lib;$Env:LIBPATH"
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46 HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	47 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	48 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	49 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	50 if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 }
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	53 Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	56 HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 $Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59 hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	60 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	62 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63 hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	66 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	67 hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	68 Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	69 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	71 BUILD_INNO = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	72 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	73 $python = "C:\hgdev\python27-{arch}\python.exe"
43561 081a77df7bc6 packaging: consolidate CLI functionality into packaging.py Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	74 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --python $python
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 '''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 C:\hgdev\python27-{arch}\Scripts\pip.exe wheel --wheel-dir dist .
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	86 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	87
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	88 BUILD_WIX = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	89 Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	90 $python = "C:\hgdev\python27-{arch}\python.exe"
43561 081a77df7bc6 packaging: consolidate CLI functionality into packaging.py Gregory Szorc <gregory.szorc@gmail.com> parents: 43076 diff changeset	91 C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py wix --python $python {extra_args}
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	92 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	93 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	94 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	95 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	96
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	97 RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	98 C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99 if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	100 throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 }}
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	102 '''
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	103
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	104 X86_WHEEL_FILENAME = 'mercurial-{version}-cp27-cp27m-win32.whl'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	105 X64_WHEEL_FILENAME = 'mercurial-{version}-cp27-cp27m-win_amd64.whl'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	106 X86_EXE_FILENAME = 'Mercurial-{version}.exe'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	107 X64_EXE_FILENAME = 'Mercurial-{version}-x64.exe'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	108 X86_MSI_FILENAME = 'mercurial-{version}-x86.msi'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	109 X64_MSI_FILENAME = 'mercurial-{version}-x64.msi'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	110
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	111 MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	112
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	113 X86_USER_AGENT_PATTERN = '.Windows.'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	114 X64_USER_AGENT_PATTERN = '.Windows.(WOW\|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	115
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	116 X86_EXE_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	117 'Mercurial {version} Inno Setup installer - x86 Windows '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	118 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	119 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	120 X64_EXE_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	121 'Mercurial {version} Inno Setup installer - x64 Windows '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	122 '- does not require admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	123 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	124 X86_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	125 'Mercurial {version} MSI installer - x86 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	126 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	127 X64_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	128 'Mercurial {version} MSI installer - x64 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	129 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	130
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	131
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	132 def get_vc_prefix(arch):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	133 if arch == 'x86':
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	134 return ACTIVATE_VC9_X86
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	135 elif arch == 'x64':
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	136 return ACTIVATE_VC9_AMD64
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	137 else:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	138 raise ValueError('illegal arch: %s; must be x86 or x64' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	139
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	140
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	141 def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	142 commands = [
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	143 '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	144 'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	145 r'icacls %s /remove:g "NT Service\sshd"' % path,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	146 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	147
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	148 run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	149
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	150
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	151 def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	152 """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	153
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154 winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	157 temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	158
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	159 ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	160 ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	161 ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163 # Generate SSH key to use for communication.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	164 subprocess.run(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	165 [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	166 'ssh-keygen',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	167 '-t',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	168 'rsa',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	169 '-b',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	170 '4096',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	171 '-N',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	172 '',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	173 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	174 str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	175 ],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	176 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	177 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	178 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180 # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	181 # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	182 authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	183 winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184 winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185 fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	186
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189 ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191 with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192 fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	193 fh.write(' User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 fh.write(' StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	195 fh.write(' UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	196 fh.write(' IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	197
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	198 if not (hg_repo / '.hg').is_dir():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	199 raise Exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	200 '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	201 'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	202 )
42308 4274b1369b75 automation: add check that hg source directory is a repo Gregory Szorc <gregory.szorc@gmail.com> parents: 42064 diff changeset	203
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	204 env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	205 env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	206 env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	208 hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	209
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210 res = subprocess.run(
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 ['python2.7', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	212 cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	213 env=env,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	214 check=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	215 capture_output=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	216 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218 full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220 args = [
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	221 'python2.7',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	222 hg_bin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	223 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	224 'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	225 '--config',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	226 'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42658 9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	227 # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags Gregory Szorc <gregory.szorc@gmail.com> parents: 42651 diff changeset	228 # calculation works.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	229 'push',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	230 '-f',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	231 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	232 full_revision,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	233 '-r',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	234 'file(.hgtags)',
42309 5c242c427897 automation: do a force push to synchronize Gregory Szorc <gregory.szorc@gmail.com> parents: 42308 diff changeset	235 'ssh://%s/c:/hgdev/src' % public_ip,
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 ]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237
42651 24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	238 res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	239
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	240 # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	241 if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push` Gregory Szorc <gregory.szorc@gmail.com> parents: 42601 diff changeset	242 res.check_returncode()
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	243
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	244 run_powershell(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	245 winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	246 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	247
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	248 # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	249
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	251 def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	252 """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	253 run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	254
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	255
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	256 def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	257 """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	258
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	259 res = winrm_client.execute_ps(
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	260 r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	261 '\| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	262 '\| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	263 '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	264 )
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	265 return res[0]
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	266
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	267
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	268 def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	269 """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	270
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	271 Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	272 matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	273 local machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	274 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	275 latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	276 source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	277 dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	278 print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	279 winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	280
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	281
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	282 def build_inno_installer(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	283 winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	284 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	285 """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	286
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	287 Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	288 a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	289 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	290 print('building Inno Setup installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	291
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292 extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293 if version:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294 extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	296 ps = get_vc_prefix(arch) + BUILD_INNO.format(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	297 arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	298 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	300 copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	302
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	303 def build_wheel(winrm_client, arch: str, dest_path: pathlib.Path):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304 """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306 Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307 for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	309 print('Building Windows wheel for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	310 ps = get_vc_prefix(arch) + BUILD_WHEEL.format(arch=arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	311 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312 copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	315 def build_wix_installer(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	316 winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	317 ):
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318 """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	319
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	320 Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	321 """
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	322 print('Building WiX installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323 extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	324 if version:
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	325 extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	326
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	327 ps = get_vc_prefix(arch) + BUILD_WIX.format(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	328 arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	329 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	330 run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	331 copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	332
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334 def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338 ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340 ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	341 """
42064 0e9066db5e44 automation: use raw strings when there are backslashes Gregory Szorc <gregory.szorc@gmail.com> parents: 42024 diff changeset	342 if not re.match(r'\d\.\d', python_version):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	343 raise ValueError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	344 r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	345 )
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	346
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347 if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	352 ps = RUN_TESTS.format(python_path=python_path, test_flags=test_flags or '',)
42024 b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353
b05a3e28cf24 automation: perform tasks on remote machines Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 run_powershell(winrm_client, ps)
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	355
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	356
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	357 def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	358 return (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	359 dist_path / X86_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	360 dist_path / X64_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	361 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	362
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	363
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	364 def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	365 return (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	366 dist_path / X86_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	367 dist_path / X64_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	368 dist_path / X86_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	369 dist_path / X64_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	370 dist_path / X86_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	371 dist_path / X64_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	372 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	373
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	374
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	375 def generate_latest_dat(version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	376 x86_exe_filename = X86_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	377 x64_exe_filename = X64_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	378 x86_msi_filename = X86_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	379 x64_msi_filename = X64_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	380
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	381 entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	382 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	383 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	384 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	385 X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	386 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	387 X86_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	388 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	389 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	390 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	391 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	392 X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	393 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	394 X64_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	395 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	396 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	397 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	398 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	399 X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	400 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_msi_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	401 X86_MSI_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	402 ),
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	403 (
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	404 '10',
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	405 version,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	406 X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	407 '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_msi_filename),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	408 X64_MSI_DESCRIPTION.format(version=version),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	409 ),
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	410 )
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	411
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	412 lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	413
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	414 return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	415
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	416
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	417 def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	418 """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	419
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	420 wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	421
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	422 for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	423 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	424 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	425
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	426 print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	427 pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	428
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	429
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	430 def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	431 dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	432 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	433 """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	434 all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	435
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	436 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	437 if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	438 raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	439
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	440 client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	441 client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	442 # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	443 print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	444 try:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	445 client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	446 except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	447 print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	448 raise
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	449
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	450 print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	451
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	452 print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	453 sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	454 print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	455
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	456 for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	457 dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	458 print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	459
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	460 with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	461 data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	462
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	463 with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	464 fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	465 fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	466
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	467 latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	468
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	469 now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	470 backup_path = dist_path / (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	471 'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	472 )
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	473 print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	474
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	475 with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	476 latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	477
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	478 with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	479 fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	480
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	481 print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	482 latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	483 print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	484
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	485 with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	486 fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	487
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	488
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	489 def publish_artifacts(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	490 dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	491 version: str,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	492 pypi=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	493 mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	494 ssh_username=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	495 ):
42913 92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	496 """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	497
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	498 Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	499 `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	500
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	501 `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	502 `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	503 """
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	504 if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	505 publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	506
92593d72e10b automation: implement "publish-windows-artifacts" command Gregory Szorc <gregory.szorc@gmail.com> parents: 42658 diff changeset	507 if mercurial_scm_org:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	508 publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	509 dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42913 diff changeset	510 )

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/automation/hgautomation/windows.py @ 43561:081a77df7bc6