contrib/automation/hgautomation/windows.py
author Gregory Szorc <gregory.szorc@gmail.com>
Thu, 05 Sep 2019 21:09:58 -0700
changeset 42907 92593d72e10b
parent 42684 9e0f1c80cddb
child 43076 2372284d9457
permissions -rw-r--r--
automation: implement "publish-windows-artifacts" command The new command and associated functionality can be used to automate the publishing of Windows release artifacts. It supports uploading wheels to PyPI (using twine) and copying the artifacts to mercurial-scm.org and updating the latest.dat file to advertise them via the website. I ran `automation.py publish-windows-artifacts 5.1.1` and it appeared to "just work." But the real test will be to do this on the next release... Differential Revision: https://phab.mercurial-scm.org/D6786
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
#
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
# Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
#
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     8
# no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    10
import datetime
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    11
import os
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    12
import paramiko
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    13
import pathlib
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    14
import re
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
import subprocess
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
import tempfile
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    17
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    18
from .pypi import (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    19
    upload as pypi_upload,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    20
)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    21
from .winrm import (
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
    run_powershell,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    23
)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    24
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
# PowerShell commands to activate a Visual Studio 2008 environment.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
# This is essentially a port of vcvarsall.bat to PowerShell.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    28
ACTIVATE_VC9_AMD64 = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    29
Write-Output "activating Visual Studio 2008 environment for AMD64"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    30
$root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    31
$Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    32
$Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    33
$Env:PATH = "${root}\VC\Bin\amd64;${root}\WinSDK\Bin\x64;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    34
$Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    35
$Env:LIB = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIB"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    36
$Env:LIBPATH = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIBPATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    37
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    38
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    39
ACTIVATE_VC9_X86 = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    40
Write-Output "activating Visual Studio 2008 environment for x86"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    41
$root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
$Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    43
$Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    44
$Env:PATH = "${root}\VC\Bin;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    45
$Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:INCLUDE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    46
$Env:LIB = "${root}\VC\Lib;${root}\WinSDK\Lib;$Env:LIB"
42601
862f6bddacce automation: correct the path separator in LIBPATH on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 42282
diff changeset
    47
$Env:LIBPATH = "${root}\VC\lib;${root}\WinSDK\Lib;$Env:LIBPATH"
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    48
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    49
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    50
HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
$Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    52
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    53
hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    55
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    56
}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    57
Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    58
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    59
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    60
HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    61
$Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    62
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    63
hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    64
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    65
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    66
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    67
hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    68
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    69
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    70
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    71
hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    72
Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    73
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    74
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    75
BUILD_INNO = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    76
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    77
$python = "C:\hgdev\python27-{arch}\python.exe"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    78
C:\hgdev\python37-x64\python.exe contrib\packaging\inno\build.py --python $python
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    79
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    80
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    81
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    82
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    84
BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    86
C:\hgdev\python27-{arch}\Scripts\pip.exe wheel --wheel-dir dist .
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    87
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    92
BUILD_WIX = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    93
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    94
$python = "C:\hgdev\python27-{arch}\python.exe"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    95
C:\hgdev\python37-x64\python.exe contrib\packaging\wix\build.py --python $python {extra_args}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    96
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    97
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    98
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   100
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   102
C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   104
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   105
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   106
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   107
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   108
X86_WHEEL_FILENAME = 'mercurial-{version}-cp27-cp27m-win32.whl'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   109
X64_WHEEL_FILENAME = 'mercurial-{version}-cp27-cp27m-win_amd64.whl'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   110
X86_EXE_FILENAME = 'Mercurial-{version}.exe'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   111
X64_EXE_FILENAME = 'Mercurial-{version}-x64.exe'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   112
X86_MSI_FILENAME = 'mercurial-{version}-x86.msi'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   113
X64_MSI_FILENAME = 'mercurial-{version}-x64.msi'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   114
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   115
MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   116
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   117
X86_USER_AGENT_PATTERN = '.*Windows.*'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   118
X64_USER_AGENT_PATTERN = '.*Windows.*(WOW|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   119
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   120
X86_EXE_DESCRIPTION = ('Mercurial {version} Inno Setup installer - x86 Windows '
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   121
                       '- does not require admin rights')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   122
X64_EXE_DESCRIPTION = ('Mercurial {version} Inno Setup installer - x64 Windows '
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   123
                       '- does not require admin rights')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   124
X86_MSI_DESCRIPTION = ('Mercurial {version} MSI installer - x86 Windows '
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   125
                       '- requires admin rights')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   126
X64_MSI_DESCRIPTION = ('Mercurial {version} MSI installer - x64 Windows '
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   127
                       '- requires admin rights')
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   128
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   129
def get_vc_prefix(arch):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   130
    if arch == 'x86':
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   131
        return ACTIVATE_VC9_X86
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   132
    elif arch == 'x64':
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   133
        return ACTIVATE_VC9_AMD64
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   134
    else:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   135
        raise ValueError('illegal arch: %s; must be x86 or x64' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   136
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   137
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   138
def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   139
    commands = [
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   140
        '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   141
        'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   142
        r'icacls %s /remove:g "NT Service\sshd"' % path,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   143
    ]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   144
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   145
    run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   146
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   147
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   148
def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   149
    """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   150
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   151
    winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   152
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   153
    with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   154
        temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   155
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   156
        ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   157
        ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   158
        ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   159
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   160
        # Generate SSH key to use for communication.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   161
        subprocess.run([
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   162
            'ssh-keygen', '-t', 'rsa', '-b', '4096', '-N', '',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   163
            '-f', str(ssh_dir / 'id_rsa')],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   164
            check=True, capture_output=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   165
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   166
        # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   167
        # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   168
        authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   169
        winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   170
        winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   171
        fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   172
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   173
        public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   174
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   175
        ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   176
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   177
        with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   178
            fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   179
            fh.write('  User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   180
            fh.write('  StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   181
            fh.write('  UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   182
            fh.write('  IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   183
42281
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   184
        if not (hg_repo / '.hg').is_dir():
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   185
            raise Exception('%s is not a Mercurial repository; '
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   186
                            'synchronization not yet supported' % hg_repo)
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   187
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   188
        env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   189
        env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   190
        env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   191
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   192
        hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   193
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   194
        res = subprocess.run(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   195
            ['python2.7', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   196
            cwd=str(hg_repo), env=env, check=True, capture_output=True)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   197
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   198
        full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   199
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   200
        args = [
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   201
            'python2.7', hg_bin,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   202
            '--config', 'ui.ssh=ssh -F %s' % ssh_config,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   203
            '--config', 'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42684
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   204
            # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   205
            # calculation works.
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   206
            'push', '-f', '-r', full_revision, '-r', 'file(.hgtags)',
42282
5c242c427897 automation: do a force push to synchronize
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42281
diff changeset
   207
            'ssh://%s/c:/hgdev/src' % public_ip,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   208
        ]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   209
42660
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   210
        res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   211
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   212
        # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   213
        if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   214
            res.check_returncode()
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   215
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   216
        run_powershell(winrm_client,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   217
                       HG_UPDATE_CLEAN.format(revision=full_revision))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   218
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   219
        # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   220
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   221
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   222
def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   223
    """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   224
    run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   225
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   226
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   227
def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   228
    """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   229
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   230
    res = winrm_client.execute_ps(
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   231
        r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   232
        '| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   233
        '| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   234
        '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   235
    )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   236
    return res[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   237
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   238
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   239
def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   240
    """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   241
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   242
    Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   243
    matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   244
    local machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   245
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   246
    latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   247
    source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   248
    dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   249
    print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   250
    winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   251
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   252
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   253
def build_inno_installer(winrm_client, arch: str, dest_path: pathlib.Path,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   254
                         version=None):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   255
    """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   256
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   257
    Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   258
    a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   259
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   260
    print('building Inno Setup installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   261
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   262
    extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   263
    if version:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   264
        extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   265
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   266
    ps = get_vc_prefix(arch) + BUILD_INNO.format(arch=arch,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   267
                                                 extra_args=' '.join(extra_args))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   268
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   269
    copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   270
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   271
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   272
def build_wheel(winrm_client, arch: str, dest_path: pathlib.Path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   273
    """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   274
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   275
    Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   276
    for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   277
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   278
    print('Building Windows wheel for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   279
    ps = get_vc_prefix(arch) + BUILD_WHEEL.format(arch=arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   280
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   281
    copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   282
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   283
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   284
def build_wix_installer(winrm_client, arch: str, dest_path: pathlib.Path,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   285
                        version=None):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   286
    """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   287
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   288
    Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   289
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   290
    print('Building WiX installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   291
    extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   292
    if version:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   293
        extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   294
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   295
    ps = get_vc_prefix(arch) + BUILD_WIX.format(arch=arch,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   296
                                                extra_args=' '.join(extra_args))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   297
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   298
    copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   299
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   300
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   301
def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   302
    """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   303
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   304
    ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   305
    ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   306
    ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   307
    ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   308
    """
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   309
    if not re.match(r'\d\.\d', python_version):
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   310
        raise ValueError(r'python_version must be \d.\d; got %s' %
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   311
                         python_version)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   312
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   313
    if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   314
        raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   315
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   316
    python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   317
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   318
    ps = RUN_TESTS.format(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   319
        python_path=python_path,
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   320
        test_flags=test_flags or '',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   321
    )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   322
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   323
    run_powershell(winrm_client, ps)
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   324
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   325
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   326
def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   327
    return (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   328
        dist_path / X86_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   329
        dist_path / X64_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   330
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   331
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   332
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   333
def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   334
    return (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   335
        dist_path / X86_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   336
        dist_path / X64_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   337
        dist_path / X86_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   338
        dist_path / X64_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   339
        dist_path / X86_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   340
        dist_path / X64_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   341
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   342
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   343
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   344
def generate_latest_dat(version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   345
    x86_exe_filename = X86_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   346
    x64_exe_filename = X64_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   347
    x86_msi_filename = X86_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   348
    x64_msi_filename = X64_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   349
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   350
    entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   351
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   352
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   353
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   354
            X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   355
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   356
            X86_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   357
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   358
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   359
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   360
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   361
            X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   362
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   363
            X64_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   364
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   365
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   366
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   367
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   368
            X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   369
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_msi_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   370
            X86_MSI_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   371
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   372
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   373
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   374
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   375
            X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   376
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_msi_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   377
            X64_MSI_DESCRIPTION.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   378
        )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   379
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   380
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   381
    lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   382
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   383
    return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   384
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   385
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   386
def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   387
    """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   388
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   389
    wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   390
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   391
    for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   392
        if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   393
            raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   394
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   395
    print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   396
    pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   397
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   398
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   399
def publish_artifacts_mercurial_scm_org(dist_path: pathlib.Path, version: str,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   400
                                        ssh_username=None):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   401
    """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   402
    all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   403
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   404
    for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   405
        if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   406
            raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   407
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   408
    client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   409
    client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   410
    # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   411
    print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   412
    try:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   413
        client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   414
    except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   415
        print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   416
        raise
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   417
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   418
    print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   419
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   420
    print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   421
    sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   422
    print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   423
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   424
    for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   425
        dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   426
        print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   427
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   428
        with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   429
            data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   430
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   431
        with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   432
            fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   433
            fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   434
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   435
    latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   436
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   437
    now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   438
    backup_path = dist_path / (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   439
        'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S'))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   440
    print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   441
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   442
    with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   443
        latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   444
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   445
    with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   446
        fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   447
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   448
    print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   449
    latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   450
    print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   451
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   452
    with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   453
        fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   454
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   455
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   456
def publish_artifacts(dist_path: pathlib.Path, version: str,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   457
                      pypi=True, mercurial_scm_org=True,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   458
                      ssh_username=None):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   459
    """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   460
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   461
    Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   462
    `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   463
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   464
    `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   465
    `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   466
    """
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   467
    if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   468
        publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   469
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   470
    if mercurial_scm_org:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   471
        publish_artifacts_mercurial_scm_org(dist_path, version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   472
                                            ssh_username=ssh_username)