mercurial/vfs.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 11 Mar 2025 10:37:39 +0100
branchstable
changeset 53041 46603c00a9f2
parent 52930 22e264ac7f60
permissions -rw-r--r--
stream: audit path on encoded path In 5b8f6e198a6e, we wrongly called `vfs._auditpath` on the pre-encoding path, so we would get abort on problem that the encoding is here to handle. So we fix the problem (easily) and update the test to cover this (with significantly more work). Thanks to Matt Harbison for spotting this during the freeze.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
31217
0f31830fbfc4 vfs: extract 'vfs' class and related code to a new 'vfs' module (API)
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31025
diff changeset
     1
# vfs.py - Mercurial 'vfs' classes
13962
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
     2
#
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
     3
#  Copyright Olivia Mackall <olivia@selenic.com>
13962
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
     4
#
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
     7
51859
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51675
diff changeset
     8
from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51675
diff changeset
     9
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
    10
import abc
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
    11
import contextlib
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    12
import os
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    13
import shutil
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    14
import stat
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
    15
import threading
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    16
import typing
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    17
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
    18
from typing import (
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    19
    Any,
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    20
    BinaryIO,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    21
    Callable,
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
    22
    Dict,
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    23
    Iterable,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    24
    Iterator,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    25
    List,
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    26
    MutableMapping,
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
    27
    Optional,
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
    28
    Set,
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    29
    Tuple,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    30
    Type,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    31
    TypeVar,
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
    32
    Union,
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
    33
)
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
    34
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    35
from .i18n import _
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    36
from . import (
34022
d5b2beca16c0 python3: wrap all uses of <exception>.strerror with strtolocal
Augie Fackler <raf@durin42.com>
parents: 33649
diff changeset
    37
    encoding,
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    38
    error,
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    39
    pathutil,
30305
af7c60988f6e py3: make scmutil.rcpath() return bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30109
diff changeset
    40
    pycompat,
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    41
    util,
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
    42
)
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents: 18678
diff changeset
    43
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    44
if typing.TYPE_CHECKING:
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    45
    from . import (
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    46
        ui as uimod,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    47
    )
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    48
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    49
    _Tbackgroundfilecloser = TypeVar(
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    50
        '_Tbackgroundfilecloser', bound='backgroundfilecloser'
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    51
    )
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    52
    _Tclosewrapbase = TypeVar('_Tclosewrapbase', bound='closewrapbase')
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    53
    _OnErrorFn = Callable[[Exception], Optional[object]]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
    54
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    55
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
    56
def _avoidambig(path: bytes, oldstat: util.filestat) -> None:
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    57
    """Avoid file stat ambiguity forcibly
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    58
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    59
    This function causes copying ``path`` file, if it is owned by
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    60
    another (see issue5418 and issue5584 for detail).
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    61
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
    62
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    63
    def checkandavoid():
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    64
        newstat = util.filestat.frompath(path)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    65
        # return whether file stat ambiguity is (already) avoided
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
    66
        return not newstat.isambig(oldstat) or newstat.avoidambig(path, oldstat)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
    67
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    68
    if not checkandavoid():
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    69
        # simply copy to change owner of path to get privilege to
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    70
        # advance mtime (see issue5418)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    71
        util.rename(util.mktempcopy(path), path)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    72
        checkandavoid()
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
    73
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
    74
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
    75
class abstractvfs(abc.ABC):
14089
d3f7e110c3c0 opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14068
diff changeset
    76
    """Abstract base class; cannot be instantiated"""
d3f7e110c3c0 opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14068
diff changeset
    77
47809
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
    78
    # default directory separator for vfs
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
    79
    #
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
    80
    # Other vfs code always use `/` and this works fine because python file API
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
    81
    # abstract the use of `/` and make it work transparently. For consistency
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
    82
    # vfs will always use `/` when joining. This avoids some confusion in
47809
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
    83
    # encoded vfs (see issue6546)
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    84
    _dir_sep: bytes = b'/'
47809
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
    85
52163
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51890
diff changeset
    86
    # Used to disable the Rust `InnerRevlog` in case the VFS is not supported
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51890
diff changeset
    87
    # by the Rust code
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51890
diff changeset
    88
    rust_compatible = True
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51890
diff changeset
    89
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
    90
    # createmode is always available on subclasses
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
    91
    createmode: int
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
    92
52930
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
    93
    _chmod: bool
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
    94
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
    95
    # TODO: type return, which is util.posixfile wrapped by a proxy
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
    96
    @abc.abstractmethod
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
    97
    def __call__(self, path: bytes, mode: bytes = b'rb', **kwargs) -> Any:
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
    98
        ...
43467
b16912f8c07c vfs: add a NotImplementedError implementation of __call__
Augie Fackler <augie@google.com>
parents: 43466
diff changeset
    99
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   100
    @abc.abstractmethod
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   101
    def _auditpath(self, path: bytes, mode: bytes) -> None:
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   102
        ...
41091
d9b6b9ed96d8 vfs: add a `_auditpath` to abstract vfs
Boris Feld <boris.feld@octobus.net>
parents: 41090
diff changeset
   103
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   104
    @abc.abstractmethod
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   105
    def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   106
        ...
43468
8b0fa4de0064 vfs: add NotImplementedError version of join
Augie Fackler <augie@google.com>
parents: 43467
diff changeset
   107
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   108
    def tryread(self, path: bytes) -> bytes:
16479
fc04698fa778 opener: coding style, use triple quotes for doc string
Thomas Arendsen Hein <thomas@intevation.de>
parents: 16455
diff changeset
   109
        '''gracefully return an empty string for missing files'''
16455
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
   110
        try:
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
   111
            return self.read(path)
49306
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
   112
        except FileNotFoundError:
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
   113
            pass
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   114
        return b""
16455
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
   115
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   116
    def tryreadlines(self, path: bytes, mode: bytes = b'rb') -> List[bytes]:
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   117
        '''gracefully return an empty array for missing files'''
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   118
        try:
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   119
            return self.readlines(path, mode=mode)
49306
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
   120
        except FileNotFoundError:
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
   121
            pass
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   122
        return []
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   123
29718
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29714
diff changeset
   124
    @util.propertycache
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29714
diff changeset
   125
    def open(self):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   126
        """Open ``path`` file, which is relative to vfs root.
23370
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
   127
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
   128
        Newly created directories are marked as "not to be indexed by
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
   129
        the content indexing service", if ``notindexed`` is specified
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
   130
        for "write" mode access.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   131
        """
29718
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29714
diff changeset
   132
        return self.__call__
19897
896a4568def7 vfs: add "open()" for newly added code paths which open files via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19896
diff changeset
   133
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   134
    def read(self, path: bytes) -> bytes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   135
        with self(path, b'rb') as fp:
14097
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
   136
            return fp.read()
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
   137
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   138
    def readlines(self, path: bytes, mode: bytes = b'rb') -> List[bytes]:
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
   139
        with self(path, mode=mode) as fp:
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   140
            return fp.readlines()
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
   141
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   142
    def write(
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   143
        self, path: bytes, data: bytes, backgroundclose: bool = False, **kwargs
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   144
    ) -> int:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   145
        with self(path, b'wb', backgroundclose=backgroundclose, **kwargs) as fp:
14167
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14138
diff changeset
   146
            return fp.write(data)
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14138
diff changeset
   147
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   148
    def writelines(
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   149
        self,
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   150
        path: bytes,
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   151
        data: Iterable[bytes],
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   152
        mode: bytes = b'wb',
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   153
        notindexed: bool = False,
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   154
    ) -> None:
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
   155
        with self(path, mode=mode, notindexed=notindexed) as fp:
23371
1df6519eb3ab vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23370
diff changeset
   156
            return fp.writelines(data)
1df6519eb3ab vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23370
diff changeset
   157
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   158
    def append(self, path: bytes, data: bytes) -> int:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   159
        with self(path, b'ab') as fp:
14097
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
   160
            return fp.write(data)
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
   161
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   162
    def basename(self, path: bytes) -> bytes:
25770
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
   163
        """return base element of a path (as os.path.basename would do)
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
   164
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
   165
        This exists to allow handling of strange encoding if needed."""
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
   166
        return os.path.basename(path)
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
   167
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   168
    def chmod(self, path: bytes, mode: int) -> None:
20086
f3df2612f3c3 vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20085
diff changeset
   169
        return os.chmod(self.join(path), mode)
f3df2612f3c3 vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20085
diff changeset
   170
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   171
    def dirname(self, path: bytes) -> bytes:
25772
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
   172
        """return dirname element of a path (as os.path.dirname would do)
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
   173
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
   174
        This exists to allow handling of strange encoding if needed."""
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
   175
        return os.path.dirname(path)
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
   176
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   177
    def exists(self, path: Optional[bytes] = None) -> bool:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   178
        return os.path.exists(self.join(path))
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   179
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   180
    def fstat(self, fp: BinaryIO) -> os.stat_result:
19899
8c3dcbbfb5de changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19897
diff changeset
   181
        return util.fstat(fp)
8c3dcbbfb5de changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19897
diff changeset
   182
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   183
    def isdir(self, path: Optional[bytes] = None) -> bool:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   184
        return os.path.isdir(self.join(path))
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   185
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   186
    def isfile(self, path: Optional[bytes] = None) -> bool:
20085
589d6bb5b18d vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20045
diff changeset
   187
        return os.path.isfile(self.join(path))
589d6bb5b18d vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20045
diff changeset
   188
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   189
    def islink(self, path: Optional[bytes] = None) -> bool:
18949
138978f20180 localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18948
diff changeset
   190
        return os.path.islink(self.join(path))
138978f20180 localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18948
diff changeset
   191
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   192
    def isfileorlink(self, path: Optional[bytes] = None) -> bool:
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   193
        """return whether path is a regular file or a symlink
27571
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   194
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   195
        Unlike isfile, this doesn't follow symlinks."""
27571
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   196
        try:
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   197
            st = self.lstat(path)
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   198
        except OSError:
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   199
            return False
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   200
        mode = st.st_mode
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   201
        return stat.S_ISREG(mode) or stat.S_ISLNK(mode)
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
   202
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   203
    def _join(self, *paths: bytes) -> bytes:
47809
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   204
        root_idx = 0
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   205
        for idx, p in enumerate(paths):
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   206
            if os.path.isabs(p) or p.startswith(self._dir_sep):
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   207
                root_idx = idx
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   208
        if root_idx != 0:
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   209
            paths = paths[root_idx:]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   210
        paths = [p for p in paths if p]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   211
        return self._dir_sep.join(paths)
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   212
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   213
    def reljoin(self, *paths: bytes) -> bytes:
23581
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
   214
        """join various elements of a path together (as os.path.join would do)
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
   215
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
   216
        The vfs base is not injected so that path stay relative. This exists
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
   217
        to allow handling of strange encoding if needed."""
47809
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   218
        return self._join(*paths)
23581
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
   219
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   220
    def split(self, path: bytes) -> Tuple[bytes, bytes]:
23582
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
   221
        """split top-most element of a path (as os.path.split would do)
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
   222
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
   223
        This exists to allow handling of strange encoding if needed."""
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
   224
        return os.path.split(path)
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
   225
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   226
    def lexists(self, path: Optional[bytes] = None) -> bool:
21563
764b691b8bda vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21111
diff changeset
   227
        return os.path.lexists(self.join(path))
764b691b8bda vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21111
diff changeset
   228
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   229
    def lstat(self, path: Optional[bytes] = None) -> os.stat_result:
19900
7c21e3398931 context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19899
diff changeset
   230
        return os.lstat(self.join(path))
7c21e3398931 context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19899
diff changeset
   231
51649
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   232
    def is_mmap_safe(self, path: Optional[bytes] = None) -> bool:
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   233
        """return True if it is safe to read a file content as mmap
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   234
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   235
        This focus on the file system aspect of such safety, the application
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   236
        logic around that file is not taken into account, so caller need to
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   237
        make sure the file won't be truncated in a way that will create SIGBUS
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   238
        on access.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   239
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   240
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   241
        The initial motivation for this logic is that if mmap is used on NFS
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   242
        and somebody deletes the mapped file (e.g. by renaming on top of it),
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   243
        then you get SIGBUS, which can be pretty disruptive: we get core dump
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   244
        reports, and the process terminates without writing to the blackbox.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   245
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
   246
        Instead, in this situation we prefer to read the file normally.
51649
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   247
        The risk of ESTALE in the middle of the read remains, but it's
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   248
        smaller because we read sooner and the error should be reported
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   249
        just as any other error.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   250
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   251
        Note that python standard library does not offer the necessary function
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   252
        to detect the file stem bits. So this detection rely on compiled bits
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   253
        and is not available in pure python.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   254
        """
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   255
        # XXX Since we already assume a vfs to address a consistent file system
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   256
        # in other location, we could determine the fstype once for the root
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   257
        # and cache that value.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   258
        fstype = util.getfstype(self.join(path))
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   259
        return fstype is not None and fstype != b'nfs'
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51590
diff changeset
   260
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   261
    def listdir(self, path: Optional[bytes] = None) -> List[bytes]:
21799
dfacdd6a111e vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21716
diff changeset
   262
        return os.listdir(self.join(path))
dfacdd6a111e vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21716
diff changeset
   263
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   264
    def makedir(self, path: Optional[bytes] = None, notindexed=True) -> None:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   265
        return util.makedir(self.join(path), notindexed)
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   266
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   267
    def makedirs(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   268
        self, path: Optional[bytes] = None, mode: Optional[int] = None
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   269
    ) -> None:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   270
        return util.makedirs(self.join(path), mode)
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   271
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   272
    def makelock(self, info: bytes, path: bytes) -> None:
20090
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
   273
        return util.makelock(info, self.join(path))
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
   274
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   275
    def mkdir(self, path: Optional[bytes] = None) -> None:
17723
ab23768746fd scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17681
diff changeset
   276
        return os.mkdir(self.join(path))
ab23768746fd scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17681
diff changeset
   277
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   278
    def mkstemp(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   279
        self,
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   280
        suffix: bytes = b'',
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   281
        prefix: bytes = b'tmp',
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   282
        dir: Optional[bytes] = None,
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   283
    ) -> Tuple[int, bytes]:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   284
        fd, name = pycompat.mkstemp(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   285
            suffix=suffix, prefix=prefix, dir=self.join(dir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   286
        )
20980
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
   287
        dname, fname = util.split(name)
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
   288
        if dir:
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
   289
            return fd, os.path.join(dir, fname)
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
   290
        else:
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
   291
            return fd, fname
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
   292
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   293
    # TODO: This doesn't match osutil.listdir().  stat=False in pure;
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   294
    #  non-optional bool in cext.  'skip' is bool if we trust cext, or bytes
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   295
    #  going by how pure uses it.  Also, cext returns a custom stat structure.
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   296
    #  from cext.osutil.pyi:
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   297
    #
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   298
    #     path: bytes, st: bool, skip: Optional[bool]
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   299
    def readdir(
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   300
        self, path: Optional[bytes] = None, stat=None, skip=None
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   301
    ) -> Any:
32208
d74b0cff94a9 osutil: proxy through util (and platform) modules (API)
Yuya Nishihara <yuya@tcha.org>
parents: 31644
diff changeset
   302
        return util.listdir(self.join(path), stat, skip)
17747
aad3bce98f76 store: invoke "osutil.listdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17726
diff changeset
   303
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   304
    def readlock(self, path: bytes) -> bytes:
20090
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
   305
        return util.readlock(self.join(path))
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
   306
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   307
    def rename(self, src: bytes, dst: bytes, checkambig: bool = False) -> None:
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   308
        """Rename from src to dst
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   309
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   310
        checkambig argument is used with util.filestat, and is useful
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   311
        only if destination file is guarded by any lock
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   312
        (e.g. repo.lock or repo.wlock).
33282
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   313
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   314
        To avoid file stat ambiguity forcibly, checkambig=True involves
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   315
        copying ``src`` file, if it is owned by another. Therefore, use
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   316
        checkambig=True only in limited cases (see also issue5418 and
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   317
        issue5584 for detail).
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   318
        """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   319
        self._auditpath(dst, b'w')
32748
ed66ec39933f vfs: create copy at renaming to avoid file stat ambiguity if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32747
diff changeset
   320
        srcpath = self.join(src)
29203
731ced087a4b vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29202
diff changeset
   321
        dstpath = self.join(dst)
51881
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51880
diff changeset
   322
        oldstat = util.filestat.frompath(dstpath) if checkambig else None
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51880
diff changeset
   323
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51880
diff changeset
   324
        util.rename(srcpath, dstpath)
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51880
diff changeset
   325
29203
731ced087a4b vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29202
diff changeset
   326
        if oldstat and oldstat.stat:
33281
6af0f023d014 vfs: replace avoiding ambiguity in abstractvfs.rename with _avoidambig
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33280
diff changeset
   327
            _avoidambig(dstpath, oldstat)
18948
2f05fa162316 localrepo: use "vfs.rename()" instead of "util.rename()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18945
diff changeset
   328
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   329
    def readlink(self, path: bytes) -> bytes:
39904
5fe0b880200e py3: convert os.readlink() path to native strings on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39464
diff changeset
   330
        return util.readlink(self.join(path))
18950
647e3b0c8751 localrepo: use "vfs.readlink()" instead of "os.readlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18949
diff changeset
   331
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   332
    def removedirs(self, path: Optional[bytes] = None) -> None:
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   333
        """Remove a leaf directory and all empty intermediate ones"""
24693
0d28b0df77ea vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24689
diff changeset
   334
        return util.removedirs(self.join(path))
0d28b0df77ea vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24689
diff changeset
   335
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   336
    def rmdir(self, path: Optional[bytes] = None) -> None:
39464
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38493
diff changeset
   337
        """Remove an empty directory."""
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38493
diff changeset
   338
        return os.rmdir(self.join(path))
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38493
diff changeset
   339
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   340
    def rmtree(
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   341
        self,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   342
        path: Optional[bytes] = None,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   343
        ignore_errors: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   344
        forcibly: bool = False,
51675
be6d8ea6d3d2 typing: add a trivial type hint to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51649
diff changeset
   345
    ) -> None:
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   346
        """Remove a directory tree recursively
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   347
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   348
        If ``forcibly``, this tries to remove READ-ONLY files, too.
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   349
        """
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   350
        if forcibly:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   351
52901
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   352
            def onexc(function: Callable, path: str, excinfo: Exception):
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   353
                # Note: str is passed here even if bytes are passed to rmtree
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   354
                # on platforms where `shutil._use_fd_functions == True`.  It is
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   355
                # bytes otherwise.  Fortunately, the methods used here accept
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   356
                # both.
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   357
                if function is not os.remove:
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   358
                    raise
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   359
                # read-only files cannot be unlinked under Windows
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   360
                s = os.stat(path)
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   361
                if (s.st_mode & stat.S_IWRITE) != 0:
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   362
                    raise
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   363
                os.chmod(path, stat.S_IMODE(s.st_mode) | stat.S_IWRITE)
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   364
                os.remove(path)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   365
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   366
        else:
52901
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   367
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   368
            def onexc(*args):
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   369
                pass
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52643
diff changeset
   370
50751
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   371
        try:
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   372
            # pytype: disable=wrong-keyword-args
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   373
            return shutil.rmtree(
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   374
                self.join(path), ignore_errors=ignore_errors, onexc=onexc
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   375
            )
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   376
            # pytype: enable=wrong-keyword-args
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   377
        except TypeError:  # onexc was introduced in Python 3.12
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   378
            return shutil.rmtree(
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   379
                self.join(path), ignore_errors=ignore_errors, onerror=onexc
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
   380
            )
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
   381
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   382
    def setflags(self, path: bytes, l: bool, x: bool) -> None:
18951
d13916a00b7e localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18950
diff changeset
   383
        return util.setflags(self.join(path), l, x)
d13916a00b7e localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18950
diff changeset
   384
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   385
    def stat(self, path: Optional[bytes] = None) -> os.stat_result:
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
   386
        return os.stat(self.join(path))
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
   387
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   388
    def unlink(self, path: Optional[bytes] = None) -> None:
19895
37c0d93fb166 bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19655
diff changeset
   389
        return util.unlink(self.join(path))
37c0d93fb166 bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19655
diff changeset
   390
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   391
    def tryunlink(self, path: Optional[bytes] = None) -> bool:
31542
fad440db3565 vfs: add tryunlink method
Ryan McElroy <rmcelroy@fb.com>
parents: 31309
diff changeset
   392
        """Attempt to remove a file, ignoring missing file errors."""
51427
187c5769a629 vfs: have tryunlink tell what it did
Georges Racinet <georges.racinet@octobus.net>
parents: 50926
diff changeset
   393
        return util.tryunlink(self.join(path))
31542
fad440db3565 vfs: add tryunlink method
Ryan McElroy <rmcelroy@fb.com>
parents: 31309
diff changeset
   394
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   395
    def unlinkpath(
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   396
        self,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   397
        path: Optional[bytes] = None,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   398
        ignoremissing: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   399
        rmdir: bool = True,
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   400
    ) -> None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   401
        return util.unlinkpath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   402
            self.join(path), ignoremissing=ignoremissing, rmdir=rmdir
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   403
        )
21716
90f9be5adade vfs: add unlinkpath to vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21563
diff changeset
   404
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   405
    # TODO: could be Tuple[float, float] too.
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   406
    def utime(
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   407
        self, path: Optional[bytes] = None, t: Optional[Tuple[int, int]] = None
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   408
    ) -> None:
19896
af03279c766a bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19895
diff changeset
   409
        return os.utime(self.join(path), t)
af03279c766a bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19895
diff changeset
   410
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   411
    def walk(
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   412
        self, path: Optional[bytes] = None, onerror: Optional[_OnErrorFn] = None
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   413
    ) -> Iterator[Tuple[bytes, List[bytes], List[bytes]]]:
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
   414
        """Yield (dirpath, dirs, files) tuple for each directory under path
24725
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   415
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   416
        ``dirpath`` is relative one from the root of this vfs. This
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   417
        uses ``os.sep`` as path separator, even you specify POSIX
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   418
        style ``path``.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   419
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   420
        "The root of this vfs" is represented as empty ``dirpath``.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   421
        """
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   422
        root = os.path.normpath(self.join(None))
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   423
        # when dirpath == root, dirpath[prefixlen:] becomes empty
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   424
        # because len(dirpath) < prefixlen.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   425
        prefixlen = len(pathutil.normasprefix(root))
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   426
        for dirpath, dirs, files in os.walk(self.join(path), onerror=onerror):
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   427
            yield (dirpath[prefixlen:], dirs, files)
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
   428
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   429
    @contextlib.contextmanager
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   430
    def backgroundclosing(
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   431
        self, ui: uimod.ui, expectedcount: int = -1
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   432
    ) -> Iterator[Optional[backgroundfilecloser]]:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   433
        """Allow files to be closed asynchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   434
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   435
        When this context manager is active, ``backgroundclose`` can be passed
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   436
        to ``__call__``/``open`` to result in the file possibly being closed
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   437
        asynchronously, on a background thread.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   438
        """
35426
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
   439
        # Sharing backgroundfilecloser between threads is complex and using
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
   440
        # multiple instances puts us at risk of running out of file descriptors
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
   441
        # only allow to use backgroundfilecloser when in main thread.
51879
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51859
diff changeset
   442
        if threading.current_thread() is not threading.main_thread():
35426
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
   443
            yield
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
   444
            return
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   445
        vfs = getattr(self, 'vfs', self)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   446
        if getattr(vfs, '_backgroundfilecloser', None):
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29373
diff changeset
   447
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   448
                _(b'can only have 1 active background file closer')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   449
            )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   450
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   451
        with backgroundfilecloser(ui, expectedcount=expectedcount) as bfc:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   452
            try:
43482
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43468
diff changeset
   453
                vfs._backgroundfilecloser = (
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43468
diff changeset
   454
                    bfc  # pytype: disable=attribute-error
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43468
diff changeset
   455
                )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   456
                yield bfc
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   457
            finally:
43482
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43468
diff changeset
   458
                vfs._backgroundfilecloser = (
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43468
diff changeset
   459
                    None  # pytype: disable=attribute-error
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43468
diff changeset
   460
                )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   461
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   462
    def register_file(self, path: bytes) -> None:
47443
9ab54aa56982 vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47178
diff changeset
   463
        """generic hook point to lets fncache steer its stew"""
9ab54aa56982 vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47178
diff changeset
   464
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   465
    def prepare_streamed_file(
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   466
        self, path: bytes, known_directories: Set[bytes]
52930
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   467
    ) -> Tuple[bytes, Optional[int]]:
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   468
        """make sure we are ready to write a file from a stream clone
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   469
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   470
        The "known_directories" variable is here to avoid trying to create the
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   471
        same directories over and over during a stream clone. It will be
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   472
        updated by this function.
52930
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   473
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   474
        return (path, mode)::
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   475
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   476
            <path> is the real file system path content should be written to,
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   477
            <mode> is the file mode that need to be set if any.
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   478
        """
53041
46603c00a9f2 stream: audit path on encoded path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52930
diff changeset
   479
        real_path = self.join(path)
46603c00a9f2 stream: audit path on encoded path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52930
diff changeset
   480
        self._auditpath(real_path, b'wb')
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   481
        self.register_file(path)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   482
        dirname, basename = util.split(real_path)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   483
        if dirname not in known_directories:
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   484
            util.makedirs(dirname, self.createmode, True)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   485
            known_directories.add(dirname)
52930
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   486
        mode = None
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   487
        if self.createmode is not None:
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   488
            mode = self.createmode & 0o666
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52929
diff changeset
   489
        return real_path, mode
52929
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52927
diff changeset
   490
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   491
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
   492
class vfs(abstractvfs):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   493
    """Operate files relative to a base directory
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   494
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   495
    This class is used to hide the details of COW semantics and
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   496
    remote file access from higher level code.
33649
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
   497
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
   498
    'cacheaudited' should be enabled only if (a) vfs object is short-lived, or
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
   499
    (b) the base directory is managed by hg and considered sort-of append-only.
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
   500
    See pathutil.pathauditor() for details.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   501
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   502
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   503
    audit: Union[pathutil.pathauditor, Callable[[bytes, Optional[bytes]], Any]]
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   504
    base: bytes
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   505
    createmode: Optional[int]
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   506
    options: Dict[bytes, Any]
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   507
    _audit: bool
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   508
    _trustnlink: Optional[bool]
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   509
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   510
    def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   511
        self,
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   512
        base: bytes,
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   513
        audit: bool = True,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   514
        cacheaudited: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   515
        expandpath: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   516
        realpath: bool = False,
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   517
    ) -> None:
18945
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
   518
        if expandpath:
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
   519
            base = util.expandpath(base)
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
   520
        if realpath:
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
   521
            base = os.path.realpath(base)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   522
        self.base = base
33259
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
   523
        self._audit = audit
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
   524
        if audit:
33649
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
   525
            self.audit = pathutil.pathauditor(self.base, cached=cacheaudited)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   526
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   527
            self.audit = lambda path, mode=None: True
33259
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
   528
        self.createmode = None
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
   529
        self._trustnlink = None
43025
3518da504303 vfs: give all vfs an options attribute by default
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41095
diff changeset
   530
        self.options = {}
17554
5450c8ad9d98 scmutil: turn opener._audit into a property, mustaudit
Bryan O'Sullivan <bryano@fb.com>
parents: 17248
diff changeset
   531
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   532
    @util.propertycache
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   533
    def _cansymlink(self) -> bool:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   534
        return util.checklink(self.base)
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   535
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
   536
    @util.propertycache
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   537
    def _chmod(self) -> bool:
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
   538
        return util.checkexec(self.base)
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
   539
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   540
    def _fixfilemode(self, name: bytes) -> None:
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
   541
        if self.createmode is None or not self._chmod:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   542
            return
25658
e93036747902 global: mass rewrite to use modern octal syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25629
diff changeset
   543
        os.chmod(name, self.createmode & 0o666)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   544
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   545
    def _auditpath(self, path: bytes, mode: bytes) -> None:
40754
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39904
diff changeset
   546
        if self._audit:
41090
c8006a25b845 vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents: 40989
diff changeset
   547
            if os.path.isabs(path) and path.startswith(self.base):
c8006a25b845 vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents: 40989
diff changeset
   548
                path = os.path.relpath(path, self.base)
40754
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39904
diff changeset
   549
            r = util.checkosfilename(path)
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39904
diff changeset
   550
            if r:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   551
                raise error.Abort(b"%s: %r" % (r, path))
40754
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39904
diff changeset
   552
            self.audit(path, mode=mode)
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39904
diff changeset
   553
49909
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   554
    def isfileorlink_checkdir(
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   555
        self,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   556
        dircache: MutableMapping[bytes, bool],
51889
22e1924e9402 typing: make `vfs.isfileorlink_checkdir()` path arg required
Matt Harbison <matt_harbison@yahoo.com>
parents: 51888
diff changeset
   557
        path: bytes,
49909
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   558
    ) -> bool:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   559
        """return True if the path is a regular file or a symlink and
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   560
        the directories along the path are "normal", that is
49912
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   561
        not symlinks or nested hg repositories.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   562
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   563
        Ignores the `_audit` setting, and checks the directories regardless.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   564
        `dircache` is used to cache the directory checks.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   565
        """
49909
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   566
        try:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   567
            for prefix in pathutil.finddirs_rev_noroot(util.localpath(path)):
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   568
                if prefix in dircache:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   569
                    res = dircache[prefix]
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   570
                else:
49912
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   571
                    res = pathutil.pathauditor._checkfs_exists(
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   572
                        self.base, prefix, path
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49909
diff changeset
   573
                    )
49909
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   574
                    dircache[prefix] = res
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   575
                if not res:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   576
                    return False
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   577
        except (OSError, error.Abort):
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   578
            return False
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   579
        return self.isfileorlink(path)
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49574
diff changeset
   580
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   581
    def __call__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   582
        self,
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   583
        path: bytes,
49574
2506c3ac73f4 vfs: make the default opener mode binary
Matt Harbison <matt_harbison@yahoo.com>
parents: 49573
diff changeset
   584
        mode: bytes = b"rb",
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   585
        atomictemp: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   586
        notindexed: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   587
        backgroundclose: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   588
        checkambig: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   589
        auditpath: bool = True,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   590
        makeparentdirs: bool = True,
52927
b6f24a92b399 vfs: add a "buffering" argument to vfs mirroring the Python one
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52901
diff changeset
   591
        buffering: int = -1,
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   592
    ) -> Any:  # TODO: should be BinaryIO if util.atomictempfile can be coersed
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   593
        """Open ``path`` file, which is relative to vfs root.
23370
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
   594
40796
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   595
        By default, parent directories are created as needed. Newly created
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   596
        directories are marked as "not to be indexed by the content indexing
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   597
        service", if ``notindexed`` is specified for "write" mode access.
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   598
        Set ``makeparentdirs=False`` to not create directories implicitly.
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   599
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   600
        If ``backgroundclose`` is passed, the file may be closed asynchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   601
        It can only be used if the ``self.backgroundclosing()`` context manager
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   602
        is active. This should only be specified if the following criteria hold:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   603
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   604
        1. There is a potential for writing thousands of files. Unless you
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   605
           are writing thousands of files, the performance benefits of
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   606
           asynchronously closing files is not realized.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   607
        2. Files are opened exactly once for the ``backgroundclosing``
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   608
           active duration and are therefore free of race conditions between
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   609
           closing a file on a background thread and reopening it. (If the
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   610
           file were opened multiple times, there could be unflushed data
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   611
           because the original file handle hasn't been flushed/closed yet.)
29202
76f1ea360c7e vfs: make atomictempfile avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29119
diff changeset
   612
44566
77d48738b8e0 vfs: fix typo in comment (remove extra "l")
Kyle Lippincott <spectral@google.com>
parents: 43506
diff changeset
   613
        ``checkambig`` argument is passed to atomictempfile (valid
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   614
        only for writing), and is useful only if target file is
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
   615
        guarded by any lock (e.g. repo.lock or repo.wlock).
33282
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   616
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   617
        To avoid file stat ambiguity forcibly, checkambig=True involves
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   618
        copying ``path`` file opened in "append" mode (e.g. for
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   619
        truncation), if it is owned by another. Therefore, use
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   620
        combination of append mode and checkambig=True only in limited
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
   621
        cases (see also issue5418 and issue5584 for detail).
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44566
diff changeset
   622
        """
33257
15e9cbe6ae49 vfs: simplify path audit disabling in stream clone
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33234
diff changeset
   623
        if auditpath:
40754
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39904
diff changeset
   624
            self._auditpath(path, mode)
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
   625
        f = self.join(path)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   626
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   627
        if b"b" not in mode:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   628
            mode += b"b"  # for that other OS
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   629
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   630
        nlink = -1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   631
        if mode not in (b'r', b'rb'):
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   632
            dirname, basename = util.split(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   633
            # If basename is empty, then the path is malformed because it points
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   634
            # to a directory. Let the posixfile() call below raise IOError.
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   635
            if basename:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   636
                if atomictemp:
40796
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   637
                    if makeparentdirs:
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   638
                        util.makedirs(dirname, self.createmode, notindexed)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   639
                    return util.atomictempfile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   640
                        f, mode, self.createmode, checkambig=checkambig
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   641
                    )
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   642
                try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   643
                    if b'w' in mode:
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   644
                        util.unlink(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   645
                        nlink = 0
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   646
                    else:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   647
                        # nlinks() may behave differently for files on Windows
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   648
                        # shares if the file is open.
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
   649
                        with util.posixfile(f):
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
   650
                            nlink = util.nlinks(f)
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
   651
                            if nlink < 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   652
                                nlink = 2  # force mktempcopy (issue1922)
49306
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
   653
                except FileNotFoundError:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   654
                    nlink = 0
40796
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   655
                    if makeparentdirs:
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40754
diff changeset
   656
                        util.makedirs(dirname, self.createmode, notindexed)
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   657
                if nlink > 0:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   658
                    if self._trustnlink is None:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   659
                        self._trustnlink = nlink > 1 or util.checknlink(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   660
                    if nlink > 1 or not self._trustnlink:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
   661
                        util.rename(util.mktempcopy(f), f)
52927
b6f24a92b399 vfs: add a "buffering" argument to vfs mirroring the Python one
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52901
diff changeset
   662
        fp = util.posixfile(f, mode, buffering=buffering)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   663
        if nlink == 0:
17763
13070de77c86 vfs: backout fchmod change from 76b73ce0ffac
Matt Mackall <mpm@selenic.com>
parents: 17752
diff changeset
   664
            self._fixfilemode(f)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   665
29996
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29995
diff changeset
   666
        if checkambig:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   667
            if mode in (b'r', b'rb'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   668
                raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   669
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   670
                        b'implementation error: mode %s is not'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   671
                        b' valid for checkambig=True'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   672
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   673
                    % mode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   674
                )
29996
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29995
diff changeset
   675
            fp = checkambigatclosing(fp)
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29995
diff changeset
   676
51879
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51859
diff changeset
   677
        if (
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51859
diff changeset
   678
            backgroundclose
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51859
diff changeset
   679
            and threading.current_thread() is threading.main_thread()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   680
        ):
43484
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   681
            if (
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   682
                not self._backgroundfilecloser  # pytype: disable=attribute-error
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   683
            ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   684
                raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   685
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   686
                        b'backgroundclose can only be used when a '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   687
                        b'backgroundclosing context manager is active'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   688
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   689
                )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   690
43484
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   691
            fp = delayclosedfile(
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   692
                fp,
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   693
                self._backgroundfilecloser,  # pytype: disable=attribute-error
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43482
diff changeset
   694
            )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   695
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   696
        return fp
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   697
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   698
    def symlink(self, src: bytes, dst: bytes) -> None:
18327
4aecdb91443c scmutil: simplify vfs.audit - drop wrapped vfs.auditor
Mads Kiilerich <mads@kiilerich.com>
parents: 18316
diff changeset
   699
        self.audit(dst)
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
   700
        linkname = self.join(dst)
31549
18b9d9b95719 vfs: use tryunlink
Ryan McElroy <rmcelroy@fb.com>
parents: 31542
diff changeset
   701
        util.tryunlink(linkname)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   702
29017
07be86828e79 util: fix race in makedirs()
Adam Simpkins <simpkins@fb.com>
parents: 28819
diff changeset
   703
        util.makedirs(os.path.dirname(linkname), self.createmode)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   704
14261
e3649bcca3f6 opener: rename _can_symlink to _cansymlink
Adrian Buehlmann <adrian@cadifra.com>
parents: 14236
diff changeset
   705
        if self._cansymlink:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   706
            try:
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   707
                os.symlink(src, linkname)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25658
diff changeset
   708
            except OSError as err:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   709
                raise OSError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   710
                    err.errno,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   711
                    _(b'could not symlink to %r: %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   712
                    % (src, encoding.strtolocal(err.strerror)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   713
                    linkname,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   714
                )
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
   715
        else:
17768
9837cafc25b1 vfs: use self.write to write symlink placeholders
Matt Mackall <mpm@selenic.com>
parents: 17763
diff changeset
   716
            self.write(dst, src)
13971
bfeaa88b875d move canonpath from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13970
diff changeset
   717
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   718
    def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
   719
        if path:
47809
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   720
            parts = [self.base, path]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   721
            parts.extend(insidef)
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47443
diff changeset
   722
            return self._join(*parts)
17681
a41fd730f230 scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents: 17675
diff changeset
   723
        else:
a41fd730f230 scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents: 17675
diff changeset
   724
            return self.base
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
   725
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   726
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   727
opener: Type[vfs] = vfs
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
   728
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   729
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   730
class proxyvfs(abstractvfs, abc.ABC):
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   731
    def __init__(self, vfs: vfs) -> None:
17845
408ded42c5ec scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents: 17821
diff changeset
   732
        self.vfs = vfs
408ded42c5ec scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents: 17821
diff changeset
   733
50365
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49912
diff changeset
   734
    @property
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   735
    def createmode(self) -> Optional[int]:
50365
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49912
diff changeset
   736
        return self.vfs.createmode
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49912
diff changeset
   737
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   738
    def _auditpath(self, path: bytes, mode: bytes) -> None:
41094
adee334d94cd vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents: 41093
diff changeset
   739
        return self.vfs._auditpath(path, mode)
adee334d94cd vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents: 41093
diff changeset
   740
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27851
diff changeset
   741
    @property
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   742
    def options(self) -> Dict[bytes, Any]:
29714
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
   743
        return self.vfs.options
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
   744
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
   745
    @options.setter
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   746
    def options(self, value: Dict[bytes, Any]) -> None:
29714
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
   747
        self.vfs.options = value
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
   748
51590
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51427
diff changeset
   749
    @property
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51427
diff changeset
   750
    def audit(self):
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51427
diff changeset
   751
        return self.vfs.audit
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51427
diff changeset
   752
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   753
41093
6498f0e03526 vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents: 41092
diff changeset
   754
class filtervfs(proxyvfs, abstractvfs):
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
   755
    '''Wrapper vfs for filtering filenames with a function.'''
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
   756
51890
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51889
diff changeset
   757
    def __init__(self, vfs: vfs, filter: Callable[[bytes], bytes]) -> None:
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 33282
diff changeset
   758
        proxyvfs.__init__(self, vfs)
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
   759
        self._filter = filter
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
   760
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   761
    # TODO: The return type should be BinaryIO
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   762
    def __call__(self, path: bytes, *args, **kwargs) -> Any:
17846
f42cf30873dc scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents: 17845
diff changeset
   763
        return self.vfs(self._filter(path), *args, **kwargs)
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
   764
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   765
    def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
   766
        if path:
24628
a0b47885a1c5 vfs: make it possible to pass multiple path elements to join
Matt Harbison <matt_harbison@yahoo.com>
parents: 24582
diff changeset
   767
            return self.vfs.join(self._filter(self.vfs.reljoin(path, *insidef)))
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
   768
        else:
17846
f42cf30873dc scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents: 17845
diff changeset
   769
            return self.vfs.join(path)
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
   770
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   771
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   772
filteropener: Type[filtervfs] = filtervfs
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
   773
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   774
41093
6498f0e03526 vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents: 41092
diff changeset
   775
class readonlyvfs(proxyvfs):
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
   776
    '''Wrapper vfs preventing any writing.'''
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
   777
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   778
    def __init__(self, vfs: vfs) -> None:
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 33282
diff changeset
   779
        proxyvfs.__init__(self, vfs)
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
   780
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   781
    # TODO: The return type should be BinaryIO
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   782
    def __call__(self, path: bytes, mode: bytes = b'rb', *args, **kw) -> Any:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   783
        if mode not in (b'r', b'rb'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   784
            raise error.Abort(_(b'this vfs is read only'))
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
   785
        return self.vfs(path, mode, *args, **kw)
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
   786
49573
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49306
diff changeset
   787
    def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
26156
a112fffdb632 scmutil.readonlyvfs: implement join
Siddharth Agarwal <sid0@fb.com>
parents: 26098
diff changeset
   788
        return self.vfs.join(path, *insidef)
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
   789
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   790
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   791
class closewrapbase(abc.ABC):
29994
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   792
    """Base class of wrapper, which hooks closing
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   793
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
   794
    Do not instantiate outside the vfs layer.
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   795
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   796
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   797
    def __init__(self, fh) -> None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43484
diff changeset
   798
        object.__setattr__(self, '_origfh', fh)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   799
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   800
    def __getattr__(self, attr: str) -> Any:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   801
        return getattr(self._origfh, attr)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   802
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   803
    def __setattr__(self, attr: str, value: Any) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   804
        return setattr(self._origfh, attr, value)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   805
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   806
    def __delattr__(self, attr: str) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   807
        return delattr(self._origfh, attr)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   808
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   809
    def __enter__(self: _Tclosewrapbase) -> _Tclosewrapbase:
40939
8d9f366b7f19 vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents: 39904
diff changeset
   810
        self._origfh.__enter__()
8d9f366b7f19 vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents: 39904
diff changeset
   811
        return self
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   812
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   813
    @abc.abstractmethod
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   814
    def __exit__(self, exc_type, exc_value, exc_tb) -> None:
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   815
        ...
29994
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   816
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   817
    @abc.abstractmethod
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   818
    def close(self) -> None:
51880
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51879
diff changeset
   819
        ...
29994
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   820
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   821
29994
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   822
class delayclosedfile(closewrapbase):
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   823
    """Proxy for a file object whose close is delayed.
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   824
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
   825
    Do not instantiate outside the vfs layer.
29994
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   826
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   827
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   828
    def __init__(self, fh, closer) -> None:
52643
5cc8deb96b48 pyupgrade: modernize calls to superclass methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 52163
diff changeset
   829
        super().__init__(fh)
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43484
diff changeset
   830
        object.__setattr__(self, '_closer', closer)
29994
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29802
diff changeset
   831
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   832
    def __exit__(self, exc_type, exc_value, exc_tb) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   833
        self._closer.close(self._origfh)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   834
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   835
    def close(self) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   836
        self._closer.close(self._origfh)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   837
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   838
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   839
class backgroundfilecloser:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   840
    """Coordinates background closing of file handles on multiple threads."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   841
51888
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51887
diff changeset
   842
    def __init__(self, ui: uimod.ui, expectedcount: int = -1) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   843
        self._running = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   844
        self._entered = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   845
        self._threads = []
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   846
        self._threadexception = None
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   847
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   848
        # Only Windows/NTFS has slow file closing. So only enable by default
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   849
        # on that platform. But allow to be enabled elsewhere for testing.
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34022
diff changeset
   850
        defaultenabled = pycompat.iswindows
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   851
        enabled = ui.configbool(b'worker', b'backgroundclose', defaultenabled)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   852
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   853
        if not enabled:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   854
            return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   855
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   856
        # There is overhead to starting and stopping the background threads.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   857
        # Don't do background processing unless the file count is large enough
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   858
        # to justify it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   859
        minfilecount = ui.configint(b'worker', b'backgroundcloseminfilecount')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   860
        # FUTURE dynamically start background threads after minfilecount closes.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   861
        # (We don't currently have any callers that don't know their file count)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   862
        if expectedcount > 0 and expectedcount < minfilecount:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   863
            return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   864
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   865
        maxqueue = ui.configint(b'worker', b'backgroundclosemaxqueue')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   866
        threadcount = ui.configint(b'worker', b'backgroundclosethreadcount')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   867
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   868
        ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   869
            b'starting %d threads for background file closing\n' % threadcount
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   870
        )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   871
37844
8fb9985382be pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35725
diff changeset
   872
        self._queue = pycompat.queue.Queue(maxsize=maxqueue)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   873
        self._running = True
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   874
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   875
        for i in range(threadcount):
43462
ffd632c224c3 vfs: another bytes-str confusion on thread name
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   876
            t = threading.Thread(target=self._worker, name='backgroundcloser')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   877
            self._threads.append(t)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   878
            t.start()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   879
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   880
    def __enter__(self: _Tbackgroundfilecloser) -> _Tbackgroundfilecloser:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   881
        self._entered = True
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   882
        return self
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   883
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   884
    def __exit__(self, exc_type, exc_value, exc_tb) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   885
        self._running = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   886
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   887
        # Wait for threads to finish closing so open files don't linger for
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   888
        # longer than lifetime of context manager.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   889
        for t in self._threads:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   890
            t.join()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   891
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   892
    def _worker(self) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   893
        """Main routine for worker thread."""
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   894
        while True:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   895
            try:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   896
                fh = self._queue.get(block=True, timeout=0.100)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   897
                # Need to catch or the thread will terminate and
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   898
                # we could orphan file descriptors.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   899
                try:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   900
                    fh.close()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   901
                except Exception as e:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   902
                    # Stash so can re-raise from main thread later.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   903
                    self._threadexception = e
37844
8fb9985382be pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35725
diff changeset
   904
            except pycompat.queue.Empty:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   905
                if not self._running:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   906
                    break
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   907
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   908
    def close(self, fh) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   909
        """Schedule a file for closing."""
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   910
        if not self._entered:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   911
            raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43090
diff changeset
   912
                _(b'can only call close() when context manager active')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   913
            )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   914
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   915
        # If a background thread encountered an exception, raise now so we fail
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
   916
        # fast. Otherwise, we may potentially go on for minutes until the error
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   917
        # is acted on.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   918
        if self._threadexception:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   919
            e = self._threadexception
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   920
            self._threadexception = None
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   921
            raise e
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   922
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   923
        # If we're not actively running, close synchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   924
        if not self._running:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   925
            fh.close()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   926
            return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   927
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
   928
        self._queue.put(fh, block=True, timeout=None)
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   929
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   930
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   931
class checkambigatclosing(closewrapbase):
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   932
    """Proxy for a file object, to avoid ambiguity of file stat
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   933
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   934
    See also util.filestat for detail about "ambiguity of file stat".
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   935
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   936
    This proxy is useful only if the target file is guarded by any
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   937
    lock (e.g. repo.lock or repo.wlock)
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   938
51882
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51881
diff changeset
   939
    Do not instantiate outside the vfs layer.
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   940
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43025
diff changeset
   941
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   942
    def __init__(self, fh) -> None:
52643
5cc8deb96b48 pyupgrade: modernize calls to superclass methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 52163
diff changeset
   943
        super().__init__(fh)
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43484
diff changeset
   944
        object.__setattr__(self, '_oldstat', util.filestat.frompath(fh.name))
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   945
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   946
    def _checkambig(self) -> None:
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   947
        oldstat = self._oldstat
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   948
        if oldstat.stat:
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
   949
            _avoidambig(self._origfh.name, oldstat)
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   950
51887
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51886
diff changeset
   951
    def __exit__(self, exc_type, exc_value, exc_tb) -> None:
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   952
        self._origfh.__exit__(exc_type, exc_value, exc_tb)
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   953
        self._checkambig()
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   954
51886
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51882
diff changeset
   955
    def close(self) -> None:
29995
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   956
        self._origfh.close()
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29994
diff changeset
   957
        self._checkambig()