Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/vfs.py @ 53018:65f6f1fe43ef

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
rhg-files: use the correct long-form flag for selecting a revision This went unnoticed forever, I guess most users pass in `-r`.
author Rapha?l Gom?s <rgomes@octobus.net>
date Wed, 26 Feb 2025 09:25:53 -0500
parents 22e264ac7f60
children 46603c00a9f2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
31227
0f31830fbfc4 vfs: extract 'vfs' class and related code to a new 'vfs' module (API)
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31045
diff changeset
1 # vfs.py - Mercurial 'vfs' classes
13962
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45957
diff changeset
3 # Copyright Olivia Mackall <olivia@selenic.com>
13962
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
4 #
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
7
51901
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51739
diff changeset
8 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51739
diff changeset
9
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
10 import abc
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
11 import contextlib
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
12 import os
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
13 import shutil
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
14 import stat
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
15 import threading
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
16 import typing
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
17
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
18 from typing import (
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
19 Any,
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
20 BinaryIO,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
21 Callable,
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
22 Dict,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
23 Iterable,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
24 Iterator,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
25 List,
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
26 MutableMapping,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
27 Optional,
52941
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
28 Set,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
29 Tuple,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
30 Type,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
31 TypeVar,
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
32 Union,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
33 )
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
34
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
35 from .i18n import _
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
36 from . import (
34040
d5b2beca16c0 python3: wrap all uses of <exception>.strerror with strtolocal
Augie Fackler <raf@durin42.com>
parents: 33647
diff changeset
37 encoding,
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
38 error,
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
39 pathutil,
30318
af7c60988f6e py3: make scmutil.rcpath() return bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30109
diff changeset
40 pycompat,
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
41 util,
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
42 )
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents: 18678
diff changeset
43
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
44 if typing.TYPE_CHECKING:
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
45 from . import (
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
46 ui as uimod,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
47 )
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
48
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
49 _Tbackgroundfilecloser = TypeVar(
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
50 '_Tbackgroundfilecloser', bound='backgroundfilecloser'
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
51 )
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
52 _Tclosewrapbase = TypeVar('_Tclosewrapbase', bound='closewrapbase')
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
53 _OnErrorFn = Callable[[Exception], Optional[object]]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
54
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
55
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
56 def _avoidambig(path: bytes, oldstat: util.filestat) -> None:
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
57 """Avoid file stat ambiguity forcibly
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
58
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
59 This function causes copying ``path`` file, if it is owned by
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
60 another (see issue5418 and issue5584 for detail).
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
61 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
62
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
63 def checkandavoid():
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
64 newstat = util.filestat.frompath(path)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
65 # return whether file stat ambiguity is (already) avoided
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
66 return not newstat.isambig(oldstat) or newstat.avoidambig(path, oldstat)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
67
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
68 if not checkandavoid():
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
69 # simply copy to change owner of path to get privilege to
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
70 # advance mtime (see issue5418)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
71 util.rename(util.mktempcopy(path), path)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
72 checkandavoid()
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
73
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
74
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
75 class abstractvfs(abc.ABC):
14089
d3f7e110c3c0 opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14068
diff changeset
76 """Abstract base class; cannot be instantiated"""
d3f7e110c3c0 opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14068
diff changeset
77
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
78 # default directory separator for vfs
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
79 #
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
80 # Other vfs code always use `/` and this works fine because python file API
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
81 # abstract the use of `/` and make it work transparently. For consistency
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
82 # vfs will always use `/` when joining. This avoids some confusion in
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
83 # encoded vfs (see issue6546)
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
84 _dir_sep: bytes = b'/'
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
85
52293
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51931
diff changeset
86 # Used to disable the Rust `InnerRevlog` in case the VFS is not supported
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51931
diff changeset
87 # by the Rust code
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51931
diff changeset
88 rust_compatible = True
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 51931
diff changeset
89
52941
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
90 # createmode is always available on subclasses
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
91 createmode: int
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
92
52942
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
93 _chmod: bool
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
94
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
95 # TODO: type return, which is util.posixfile wrapped by a proxy
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
96 @abc.abstractmethod
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
97 def __call__(self, path: bytes, mode: bytes = b'rb', **kwargs) -> Any:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
98 ...
43515
b16912f8c07c vfs: add a NotImplementedError implementation of __call__
Augie Fackler <augie@google.com>
parents: 43514
diff changeset
99
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
100 @abc.abstractmethod
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
101 def _auditpath(self, path: bytes, mode: bytes) -> None:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
102 ...
41094
d9b6b9ed96d8 vfs: add a `_auditpath` to abstract vfs
Boris Feld <boris.feld@octobus.net>
parents: 41093
diff changeset
103
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
104 @abc.abstractmethod
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
105 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
106 ...
43516
8b0fa4de0064 vfs: add NotImplementedError version of join
Augie Fackler <augie@google.com>
parents: 43515
diff changeset
107
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
108 def tryread(self, path: bytes) -> bytes:
16479
fc04698fa778 opener: coding style, use triple quotes for doc string
Thomas Arendsen Hein <thomas@intevation.de>
parents: 16455
diff changeset
109 '''gracefully return an empty string for missing files'''
16455
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
110 try:
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
111 return self.read(path)
49314
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
112 except FileNotFoundError:
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
113 pass
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
114 return b""
16455
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
115
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
116 def tryreadlines(self, path: bytes, mode: bytes = b'rb') -> List[bytes]:
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
117 '''gracefully return an empty array for missing files'''
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
118 try:
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
119 return self.readlines(path, mode=mode)
49314
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
120 except FileNotFoundError:
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
121 pass
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
122 return []
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
123
29730
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29727
diff changeset
124 @util.propertycache
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29727
diff changeset
125 def open(self):
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
126 """Open ``path`` file, which is relative to vfs root.
23370
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
127
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
128 Newly created directories are marked as "not to be indexed by
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
129 the content indexing service", if ``notindexed`` is specified
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
130 for "write" mode access.
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
131 """
29730
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29727
diff changeset
132 return self.__call__
19897
896a4568def7 vfs: add "open()" for newly added code paths which open files via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19896
diff changeset
133
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
134 def read(self, path: bytes) -> bytes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
135 with self(path, b'rb') as fp:
14097
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
136 return fp.read()
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
137
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
138 def readlines(self, path: bytes, mode: bytes = b'rb') -> List[bytes]:
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
139 with self(path, mode=mode) as fp:
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
140 return fp.readlines()
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
141
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
142 def write(
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
143 self, path: bytes, data: bytes, backgroundclose: bool = False, **kwargs
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
144 ) -> int:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
145 with self(path, b'wb', backgroundclose=backgroundclose, **kwargs) as fp:
14167
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14138
diff changeset
146 return fp.write(data)
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14138
diff changeset
147
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
148 def writelines(
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
149 self,
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
150 path: bytes,
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
151 data: Iterable[bytes],
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
152 mode: bytes = b'wb',
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
153 notindexed: bool = False,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
154 ) -> None:
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
155 with self(path, mode=mode, notindexed=notindexed) as fp:
23371
1df6519eb3ab vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23370
diff changeset
156 return fp.writelines(data)
1df6519eb3ab vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23370
diff changeset
157
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
158 def append(self, path: bytes, data: bytes) -> int:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
159 with self(path, b'ab') as fp:
14097
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
160 return fp.write(data)
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
161
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
162 def basename(self, path: bytes) -> bytes:
25770
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
163 """return base element of a path (as os.path.basename would do)
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
164
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
165 This exists to allow handling of strange encoding if needed."""
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
166 return os.path.basename(path)
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
167
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
168 def chmod(self, path: bytes, mode: int) -> None:
20086
f3df2612f3c3 vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20085
diff changeset
169 return os.chmod(self.join(path), mode)
f3df2612f3c3 vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20085
diff changeset
170
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
171 def dirname(self, path: bytes) -> bytes:
25772
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
172 """return dirname element of a path (as os.path.dirname would do)
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
173
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
174 This exists to allow handling of strange encoding if needed."""
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
175 return os.path.dirname(path)
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
176
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
177 def exists(self, path: Optional[bytes] = None) -> bool:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
178 return os.path.exists(self.join(path))
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
179
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
180 def fstat(self, fp: BinaryIO) -> os.stat_result:
19899
8c3dcbbfb5de changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19897
diff changeset
181 return util.fstat(fp)
8c3dcbbfb5de changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19897
diff changeset
182
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
183 def isdir(self, path: Optional[bytes] = None) -> bool:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
184 return os.path.isdir(self.join(path))
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
185
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
186 def isfile(self, path: Optional[bytes] = None) -> bool:
20085
589d6bb5b18d vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20045
diff changeset
187 return os.path.isfile(self.join(path))
589d6bb5b18d vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20045
diff changeset
188
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
189 def islink(self, path: Optional[bytes] = None) -> bool:
18949
138978f20180 localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18948
diff changeset
190 return os.path.islink(self.join(path))
138978f20180 localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18948
diff changeset
191
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
192 def isfileorlink(self, path: Optional[bytes] = None) -> bool:
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
193 """return whether path is a regular file or a symlink
27571
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
194
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
195 Unlike isfile, this doesn't follow symlinks."""
27571
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
196 try:
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
197 st = self.lstat(path)
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
198 except OSError:
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
199 return False
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
200 mode = st.st_mode
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
201 return stat.S_ISREG(mode) or stat.S_ISLNK(mode)
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
202
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
203 def _join(self, *paths: bytes) -> bytes:
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
204 root_idx = 0
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
205 for idx, p in enumerate(paths):
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
206 if os.path.isabs(p) or p.startswith(self._dir_sep):
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
207 root_idx = idx
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
208 if root_idx != 0:
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
209 paths = paths[root_idx:]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
210 paths = [p for p in paths if p]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
211 return self._dir_sep.join(paths)
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
212
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
213 def reljoin(self, *paths: bytes) -> bytes:
23581
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
214 """join various elements of a path together (as os.path.join would do)
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
215
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
216 The vfs base is not injected so that path stay relative. This exists
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
217 to allow handling of strange encoding if needed."""
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
218 return self._join(*paths)
23581
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
219
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
220 def split(self, path: bytes) -> Tuple[bytes, bytes]:
23582
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
221 """split top-most element of a path (as os.path.split would do)
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
222
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
223 This exists to allow handling of strange encoding if needed."""
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
224 return os.path.split(path)
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
225
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
226 def lexists(self, path: Optional[bytes] = None) -> bool:
21563
764b691b8bda vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21111
diff changeset
227 return os.path.lexists(self.join(path))
764b691b8bda vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21111
diff changeset
228
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
229 def lstat(self, path: Optional[bytes] = None) -> os.stat_result:
19900
7c21e3398931 context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19899
diff changeset
230 return os.lstat(self.join(path))
7c21e3398931 context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19899
diff changeset
231
51649
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
232 def is_mmap_safe(self, path: Optional[bytes] = None) -> bool:
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
233 """return True if it is safe to read a file content as mmap
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
234
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
235 This focus on the file system aspect of such safety, the application
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
236 logic around that file is not taken into account, so caller need to
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
237 make sure the file won't be truncated in a way that will create SIGBUS
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
238 on access.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
239
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
240
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
241 The initial motivation for this logic is that if mmap is used on NFS
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
242 and somebody deletes the mapped file (e.g. by renaming on top of it),
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
243 then you get SIGBUS, which can be pretty disruptive: we get core dump
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
244 reports, and the process terminates without writing to the blackbox.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
245
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
246 Instead, in this situation we prefer to read the file normally.
51649
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
247 The risk of ESTALE in the middle of the read remains, but it's
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
248 smaller because we read sooner and the error should be reported
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
249 just as any other error.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
250
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
251 Note that python standard library does not offer the necessary function
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
252 to detect the file stem bits. So this detection rely on compiled bits
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
253 and is not available in pure python.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
254 """
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
255 # XXX Since we already assume a vfs to address a consistent file system
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
256 # in other location, we could determine the fstype once for the root
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
257 # and cache that value.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
258 fstype = util.getfstype(self.join(path))
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
259 return fstype is not None and fstype != b'nfs'
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
260
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
261 def listdir(self, path: Optional[bytes] = None) -> List[bytes]:
21799
dfacdd6a111e vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21716
diff changeset
262 return os.listdir(self.join(path))
dfacdd6a111e vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21716
diff changeset
263
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
264 def makedir(self, path: Optional[bytes] = None, notindexed=True) -> None:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
265 return util.makedir(self.join(path), notindexed)
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
266
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
267 def makedirs(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
268 self, path: Optional[bytes] = None, mode: Optional[int] = None
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
269 ) -> None:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
270 return util.makedirs(self.join(path), mode)
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
271
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
272 def makelock(self, info: bytes, path: bytes) -> None:
20090
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
273 return util.makelock(info, self.join(path))
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
274
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
275 def mkdir(self, path: Optional[bytes] = None) -> None:
17723
ab23768746fd scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17681
diff changeset
276 return os.mkdir(self.join(path))
ab23768746fd scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17681
diff changeset
277
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
278 def mkstemp(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
279 self,
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
280 suffix: bytes = b'',
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
281 prefix: bytes = b'tmp',
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
282 dir: Optional[bytes] = None,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
283 ) -> Tuple[int, bytes]:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
284 fd, name = pycompat.mkstemp(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
285 suffix=suffix, prefix=prefix, dir=self.join(dir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
286 )
20980
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
287 dname, fname = util.split(name)
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
288 if dir:
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
289 return fd, os.path.join(dir, fname)
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
290 else:
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
291 return fd, fname
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
292
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
293 # TODO: This doesn't match osutil.listdir(). stat=False in pure;
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
294 # non-optional bool in cext. 'skip' is bool if we trust cext, or bytes
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
295 # going by how pure uses it. Also, cext returns a custom stat structure.
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
296 # from cext.osutil.pyi:
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
297 #
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
298 # path: bytes, st: bool, skip: Optional[bool]
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
299 def readdir(
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
300 self, path: Optional[bytes] = None, stat=None, skip=None
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
301 ) -> Any:
32248
d74b0cff94a9 osutil: proxy through util (and platform) modules (API)
Yuya Nishihara <yuya@tcha.org>
parents: 31649
diff changeset
302 return util.listdir(self.join(path), stat, skip)
17747
aad3bce98f76 store: invoke "osutil.listdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17726
diff changeset
303
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
304 def readlock(self, path: bytes) -> bytes:
20090
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
305 return util.readlock(self.join(path))
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
306
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
307 def rename(self, src: bytes, dst: bytes, checkambig: bool = False) -> None:
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
308 """Rename from src to dst
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
309
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
310 checkambig argument is used with util.filestat, and is useful
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
311 only if destination file is guarded by any lock
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
312 (e.g. repo.lock or repo.wlock).
33282
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
313
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
314 To avoid file stat ambiguity forcibly, checkambig=True involves
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
315 copying ``src`` file, if it is owned by another. Therefore, use
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
316 checkambig=True only in limited cases (see also issue5418 and
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
317 issue5584 for detail).
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
318 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
319 self._auditpath(dst, b'w')
32768
ed66ec39933f vfs: create copy at renaming to avoid file stat ambiguity if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32767
diff changeset
320 srcpath = self.join(src)
29203
731ced087a4b vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29202
diff changeset
321 dstpath = self.join(dst)
51922
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
322 oldstat = util.filestat.frompath(dstpath) if checkambig else None
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
323
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
324 util.rename(srcpath, dstpath)
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
325
29203
731ced087a4b vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29202
diff changeset
326 if oldstat and oldstat.stat:
33281
6af0f023d014 vfs: replace avoiding ambiguity in abstractvfs.rename with _avoidambig
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33280
diff changeset
327 _avoidambig(dstpath, oldstat)
18948
2f05fa162316 localrepo: use "vfs.rename()" instead of "util.rename()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18945
diff changeset
328
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
329 def readlink(self, path: bytes) -> bytes:
39909
5fe0b880200e py3: convert os.readlink() path to native strings on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39481
diff changeset
330 return util.readlink(self.join(path))
18950
647e3b0c8751 localrepo: use "vfs.readlink()" instead of "os.readlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18949
diff changeset
331
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
332 def removedirs(self, path: Optional[bytes] = None) -> None:
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
333 """Remove a leaf directory and all empty intermediate ones"""
24693
0d28b0df77ea vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24689
diff changeset
334 return util.removedirs(self.join(path))
0d28b0df77ea vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24689
diff changeset
335
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
336 def rmdir(self, path: Optional[bytes] = None) -> None:
39481
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38497
diff changeset
337 """Remove an empty directory."""
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38497
diff changeset
338 return os.rmdir(self.join(path))
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38497
diff changeset
339
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
340 def rmtree(
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
341 self,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
342 path: Optional[bytes] = None,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
343 ignore_errors: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
344 forcibly: bool = False,
51739
be6d8ea6d3d2 typing: add a trivial type hint to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51649
diff changeset
345 ) -> None:
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
346 """Remove a directory tree recursively
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
347
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
348 If ``forcibly``, this tries to remove READ-ONLY files, too.
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
349 """
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
350 if forcibly:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
351
52913
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
352 def onexc(function: Callable, path: str, excinfo: Exception):
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
353 # Note: str is passed here even if bytes are passed to rmtree
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
354 # on platforms where `shutil._use_fd_functions == True`. It is
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
355 # bytes otherwise. Fortunately, the methods used here accept
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
356 # both.
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
357 if function is not os.remove:
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
358 raise
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
359 # read-only files cannot be unlinked under Windows
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
360 s = os.stat(path)
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
361 if (s.st_mode & stat.S_IWRITE) != 0:
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
362 raise
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
363 os.chmod(path, stat.S_IMODE(s.st_mode) | stat.S_IWRITE)
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
364 os.remove(path)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
365
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
366 else:
52913
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
367
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
368 def onexc(*args):
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
369 pass
6ee3c401882b vfs: avoid passing None to `shutil.rmtree()` for the exception handler
Matt Harbison <matt_harbison@yahoo.com>
parents: 52668
diff changeset
370
50750
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
371 try:
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
372 # pytype: disable=wrong-keyword-args
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
373 return shutil.rmtree(
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
374 self.join(path), ignore_errors=ignore_errors, onexc=onexc
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
375 )
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
376 # pytype: enable=wrong-keyword-args
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
377 except TypeError: # onexc was introduced in Python 3.12
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
378 return shutil.rmtree(
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
379 self.join(path), ignore_errors=ignore_errors, onerror=onexc
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
380 )
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
381
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
382 def setflags(self, path: bytes, l: bool, x: bool) -> None:
18951
d13916a00b7e localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18950
diff changeset
383 return util.setflags(self.join(path), l, x)
d13916a00b7e localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18950
diff changeset
384
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
385 def stat(self, path: Optional[bytes] = None) -> os.stat_result:
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
386 return os.stat(self.join(path))
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
387
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
388 def unlink(self, path: Optional[bytes] = None) -> None:
19895
37c0d93fb166 bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19655
diff changeset
389 return util.unlink(self.join(path))
37c0d93fb166 bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19655
diff changeset
390
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
391 def tryunlink(self, path: Optional[bytes] = None) -> bool:
31548
fad440db3565 vfs: add tryunlink method
Ryan McElroy <rmcelroy@fb.com>
parents: 31318
diff changeset
392 """Attempt to remove a file, ignoring missing file errors."""
51428
187c5769a629 vfs: have tryunlink tell what it did
Georges Racinet <georges.racinet@octobus.net>
parents: 50952
diff changeset
393 return util.tryunlink(self.join(path))
31548
fad440db3565 vfs: add tryunlink method
Ryan McElroy <rmcelroy@fb.com>
parents: 31318
diff changeset
394
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
395 def unlinkpath(
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
396 self,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
397 path: Optional[bytes] = None,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
398 ignoremissing: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
399 rmdir: bool = True,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
400 ) -> None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
401 return util.unlinkpath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
402 self.join(path), ignoremissing=ignoremissing, rmdir=rmdir
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
403 )
21716
90f9be5adade vfs: add unlinkpath to vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21563
diff changeset
404
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
405 # TODO: could be Tuple[float, float] too.
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
406 def utime(
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
407 self, path: Optional[bytes] = None, t: Optional[Tuple[int, int]] = None
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
408 ) -> None:
19896
af03279c766a bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19895
diff changeset
409 return os.utime(self.join(path), t)
af03279c766a bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19895
diff changeset
410
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
411 def walk(
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
412 self, path: Optional[bytes] = None, onerror: Optional[_OnErrorFn] = None
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
413 ) -> Iterator[Tuple[bytes, List[bytes], List[bytes]]]:
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
414 """Yield (dirpath, dirs, files) tuple for each directory under path
24725
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
415
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
416 ``dirpath`` is relative one from the root of this vfs. This
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
417 uses ``os.sep`` as path separator, even you specify POSIX
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
418 style ``path``.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
419
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
420 "The root of this vfs" is represented as empty ``dirpath``.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
421 """
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
422 root = os.path.normpath(self.join(None))
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
423 # when dirpath == root, dirpath[prefixlen:] becomes empty
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
424 # because len(dirpath) < prefixlen.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
425 prefixlen = len(pathutil.normasprefix(root))
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
426 for dirpath, dirs, files in os.walk(self.join(path), onerror=onerror):
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
427 yield (dirpath[prefixlen:], dirs, files)
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
428
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
429 @contextlib.contextmanager
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
430 def backgroundclosing(
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
431 self, ui: uimod.ui, expectedcount: int = -1
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
432 ) -> Iterator[Optional[backgroundfilecloser]]:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
433 """Allow files to be closed asynchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
434
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
435 When this context manager is active, ``backgroundclose`` can be passed
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
436 to ``__call__``/``open`` to result in the file possibly being closed
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
437 asynchronously, on a background thread.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
438 """
35432
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
439 # Sharing backgroundfilecloser between threads is complex and using
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
440 # multiple instances puts us at risk of running out of file descriptors
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
441 # only allow to use backgroundfilecloser when in main thread.
51920
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
442 if threading.current_thread() is not threading.main_thread():
35432
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
443 yield
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
444 return
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
445 vfs = getattr(self, 'vfs', self)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
446 if getattr(vfs, '_backgroundfilecloser', None):
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29373
diff changeset
447 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
448 _(b'can only have 1 active background file closer')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
449 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
450
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
451 with backgroundfilecloser(ui, expectedcount=expectedcount) as bfc:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
452 try:
43530
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
453 vfs._backgroundfilecloser = (
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
454 bfc # pytype: disable=attribute-error
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
455 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
456 yield bfc
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
457 finally:
43530
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
458 vfs._backgroundfilecloser = (
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
459 None # pytype: disable=attribute-error
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
460 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
461
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
462 def register_file(self, path: bytes) -> None:
47449
9ab54aa56982 vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47192
diff changeset
463 """generic hook point to lets fncache steer its stew"""
9ab54aa56982 vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47192
diff changeset
464
52941
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
465 def prepare_streamed_file(
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
466 self, path: bytes, known_directories: Set[bytes]
52942
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
467 ) -> Tuple[bytes, Optional[int]]:
52941
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
468 """make sure we are ready to write a file from a stream clone
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
469
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
470 The "known_directories" variable is here to avoid trying to create the
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
471 same directories over and over during a stream clone. It will be
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
472 updated by this function.
52942
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
473
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
474 return (path, mode)::
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
475
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
476 <path> is the real file system path content should be written to,
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
477 <mode> is the file mode that need to be set if any.
52941
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
478 """
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
479 self._auditpath(path, b'wb')
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
480 self.register_file(path)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
481 real_path = self.join(path)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
482 dirname, basename = util.split(real_path)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
483 if dirname not in known_directories:
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
484 util.makedirs(dirname, self.createmode, True)
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
485 known_directories.add(dirname)
52942
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
486 mode = None
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
487 if self.createmode is not None:
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
488 mode = self.createmode & 0o666
22e264ac7f60 stream-clone-v2: bypass the vfs to write the file on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52941
diff changeset
489 return real_path, mode
52941
5b8f6e198a6e stream-clone-v2: centralize preparation for streamed files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52939
diff changeset
490
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
491
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
492 class vfs(abstractvfs):
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
493 """Operate files relative to a base directory
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
494
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
495 This class is used to hide the details of COW semantics and
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
496 remote file access from higher level code.
33647
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
497
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
498 'cacheaudited' should be enabled only if (a) vfs object is short-lived, or
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
499 (b) the base directory is managed by hg and considered sort-of append-only.
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
500 See pathutil.pathauditor() for details.
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
501 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
502
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
503 audit: Union[pathutil.pathauditor, Callable[[bytes, Optional[bytes]], Any]]
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
504 base: bytes
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
505 createmode: Optional[int]
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
506 options: Dict[bytes, Any]
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
507 _audit: bool
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
508 _trustnlink: Optional[bool]
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
509
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
510 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
511 self,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
512 base: bytes,
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
513 audit: bool = True,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
514 cacheaudited: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
515 expandpath: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
516 realpath: bool = False,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
517 ) -> None:
18945
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
518 if expandpath:
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
519 base = util.expandpath(base)
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
520 if realpath:
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
521 base = os.path.realpath(base)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
522 self.base = base
33259
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
523 self._audit = audit
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
524 if audit:
33647
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
525 self.audit = pathutil.pathauditor(self.base, cached=cacheaudited)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
526 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
527 self.audit = lambda path, mode=None: True
33259
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
528 self.createmode = None
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
529 self._trustnlink = None
43027
3518da504303 vfs: give all vfs an options attribute by default
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41098
diff changeset
530 self.options = {}
17554
5450c8ad9d98 scmutil: turn opener._audit into a property, mustaudit
Bryan O'Sullivan <bryano@fb.com>
parents: 17248
diff changeset
531
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
532 @util.propertycache
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
533 def _cansymlink(self) -> bool:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
534 return util.checklink(self.base)
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
535
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
536 @util.propertycache
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
537 def _chmod(self) -> bool:
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
538 return util.checkexec(self.base)
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
539
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
540 def _fixfilemode(self, name: bytes) -> None:
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
541 if self.createmode is None or not self._chmod:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
542 return
25658
e93036747902 global: mass rewrite to use modern octal syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25629
diff changeset
543 os.chmod(name, self.createmode & 0o666)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
544
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
545 def _auditpath(self, path: bytes, mode: bytes) -> None:
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
546 if self._audit:
41093
c8006a25b845 vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents: 40999
diff changeset
547 if os.path.isabs(path) and path.startswith(self.base):
c8006a25b845 vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents: 40999
diff changeset
548 path = os.path.relpath(path, self.base)
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
549 r = util.checkosfilename(path)
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
550 if r:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
551 raise error.Abort(b"%s: %r" % (r, path))
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
552 self.audit(path, mode=mode)
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
553
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
554 def isfileorlink_checkdir(
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
555 self,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
556 dircache: MutableMapping[bytes, bool],
51930
22e1924e9402 typing: make `vfs.isfileorlink_checkdir()` path arg required
Matt Harbison <matt_harbison@yahoo.com>
parents: 51929
diff changeset
557 path: bytes,
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
558 ) -> bool:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
559 """return True if the path is a regular file or a symlink and
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
560 the directories along the path are "normal", that is
49985
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
561 not symlinks or nested hg repositories.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
562
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
563 Ignores the `_audit` setting, and checks the directories regardless.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
564 `dircache` is used to cache the directory checks.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
565 """
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
566 try:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
567 for prefix in pathutil.finddirs_rev_noroot(util.localpath(path)):
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
568 if prefix in dircache:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
569 res = dircache[prefix]
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
570 else:
49985
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
571 res = pathutil.pathauditor._checkfs_exists(
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
572 self.base, prefix, path
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
573 )
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
574 dircache[prefix] = res
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
575 if not res:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
576 return False
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
577 except (OSError, error.Abort):
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
578 return False
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
579 return self.isfileorlink(path)
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
580
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
581 def __call__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
582 self,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
583 path: bytes,
49723
2506c3ac73f4 vfs: make the default opener mode binary
Matt Harbison <matt_harbison@yahoo.com>
parents: 49722
diff changeset
584 mode: bytes = b"rb",
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
585 atomictemp: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
586 notindexed: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
587 backgroundclose: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
588 checkambig: bool = False,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
589 auditpath: bool = True,
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
590 makeparentdirs: bool = True,
52939
b6f24a92b399 vfs: add a "buffering" argument to vfs mirroring the Python one
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52913
diff changeset
591 buffering: int = -1,
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
592 ) -> Any: # TODO: should be BinaryIO if util.atomictempfile can be coersed
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
593 """Open ``path`` file, which is relative to vfs root.
23370
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
594
40825
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
595 By default, parent directories are created as needed. Newly created
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
596 directories are marked as "not to be indexed by the content indexing
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
597 service", if ``notindexed`` is specified for "write" mode access.
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
598 Set ``makeparentdirs=False`` to not create directories implicitly.
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
599
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
600 If ``backgroundclose`` is passed, the file may be closed asynchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
601 It can only be used if the ``self.backgroundclosing()`` context manager
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
602 is active. This should only be specified if the following criteria hold:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
603
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
604 1. There is a potential for writing thousands of files. Unless you
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
605 are writing thousands of files, the performance benefits of
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
606 asynchronously closing files is not realized.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
607 2. Files are opened exactly once for the ``backgroundclosing``
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
608 active duration and are therefore free of race conditions between
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
609 closing a file on a background thread and reopening it. (If the
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
610 file were opened multiple times, there could be unflushed data
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
611 because the original file handle hasn't been flushed/closed yet.)
29202
76f1ea360c7e vfs: make atomictempfile avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29119
diff changeset
612
44575
77d48738b8e0 vfs: fix typo in comment (remove extra "l")
Kyle Lippincott <spectral@google.com>
parents: 43554
diff changeset
613 ``checkambig`` argument is passed to atomictempfile (valid
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
614 only for writing), and is useful only if target file is
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
615 guarded by any lock (e.g. repo.lock or repo.wlock).
33282
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
616
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
617 To avoid file stat ambiguity forcibly, checkambig=True involves
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
618 copying ``path`` file opened in "append" mode (e.g. for
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
619 truncation), if it is owned by another. Therefore, use
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
620 combination of append mode and checkambig=True only in limited
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
621 cases (see also issue5418 and issue5584 for detail).
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
622 """
33257
15e9cbe6ae49 vfs: simplify path audit disabling in stream clone
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33234
diff changeset
623 if auditpath:
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
624 self._auditpath(path, mode)
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
625 f = self.join(path)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
626
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
627 if b"b" not in mode:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
628 mode += b"b" # for that other OS
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
629
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
630 nlink = -1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
631 if mode not in (b'r', b'rb'):
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
632 dirname, basename = util.split(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
633 # If basename is empty, then the path is malformed because it points
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
634 # to a directory. Let the posixfile() call below raise IOError.
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
635 if basename:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
636 if atomictemp:
40825
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
637 if makeparentdirs:
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
638 util.makedirs(dirname, self.createmode, notindexed)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
639 return util.atomictempfile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
640 f, mode, self.createmode, checkambig=checkambig
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
641 )
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
642 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
643 if b'w' in mode:
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
644 util.unlink(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
645 nlink = 0
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
646 else:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
647 # nlinks() may behave differently for files on Windows
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
648 # shares if the file is open.
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
649 with util.posixfile(f):
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
650 nlink = util.nlinks(f)
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
651 if nlink < 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
652 nlink = 2 # force mktempcopy (issue1922)
49314
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
653 except FileNotFoundError:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
654 nlink = 0
40825
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
655 if makeparentdirs:
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
656 util.makedirs(dirname, self.createmode, notindexed)
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
657 if nlink > 0:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
658 if self._trustnlink is None:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
659 self._trustnlink = nlink > 1 or util.checknlink(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
660 if nlink > 1 or not self._trustnlink:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
661 util.rename(util.mktempcopy(f), f)
52939
b6f24a92b399 vfs: add a "buffering" argument to vfs mirroring the Python one
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 52913
diff changeset
662 fp = util.posixfile(f, mode, buffering=buffering)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
663 if nlink == 0:
17763
13070de77c86 vfs: backout fchmod change from 76b73ce0ffac
Matt Mackall <mpm@selenic.com>
parents: 17752
diff changeset
664 self._fixfilemode(f)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
665
30000
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29999
diff changeset
666 if checkambig:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
667 if mode in (b'r', b'rb'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
668 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
669 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
670 b'implementation error: mode %s is not'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
671 b' valid for checkambig=True'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
672 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
673 % mode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
674 )
30000
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29999
diff changeset
675 fp = checkambigatclosing(fp)
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29999
diff changeset
676
51920
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
677 if (
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
678 backgroundclose
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
679 and threading.current_thread() is threading.main_thread()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
680 ):
43532
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
681 if (
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
682 not self._backgroundfilecloser # pytype: disable=attribute-error
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
683 ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
684 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
685 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
686 b'backgroundclose can only be used when a '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
687 b'backgroundclosing context manager is active'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
688 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
689 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
690
43532
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
691 fp = delayclosedfile(
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
692 fp,
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
693 self._backgroundfilecloser, # pytype: disable=attribute-error
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
694 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
695
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
696 return fp
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
697
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
698 def symlink(self, src: bytes, dst: bytes) -> None:
18327
4aecdb91443c scmutil: simplify vfs.audit - drop wrapped vfs.auditor
Mads Kiilerich <mads@kiilerich.com>
parents: 18316
diff changeset
699 self.audit(dst)
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
700 linkname = self.join(dst)
31555
18b9d9b95719 vfs: use tryunlink
Ryan McElroy <rmcelroy@fb.com>
parents: 31548
diff changeset
701 util.tryunlink(linkname)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
702
29017
07be86828e79 util: fix race in makedirs()
Adam Simpkins <simpkins@fb.com>
parents: 28819
diff changeset
703 util.makedirs(os.path.dirname(linkname), self.createmode)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
704
14261
e3649bcca3f6 opener: rename _can_symlink to _cansymlink
Adrian Buehlmann <adrian@cadifra.com>
parents: 14236
diff changeset
705 if self._cansymlink:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
706 try:
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
707 os.symlink(src, linkname)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25658
diff changeset
708 except OSError as err:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
709 raise OSError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
710 err.errno,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
711 _(b'could not symlink to %r: %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
712 % (src, encoding.strtolocal(err.strerror)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
713 linkname,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
714 )
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
715 else:
17768
9837cafc25b1 vfs: use self.write to write symlink placeholders
Matt Mackall <mpm@selenic.com>
parents: 17763
diff changeset
716 self.write(dst, src)
13971
bfeaa88b875d move canonpath from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13970
diff changeset
717
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
718 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
719 if path:
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
720 parts = [self.base, path]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
721 parts.extend(insidef)
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
722 return self._join(*parts)
17681
a41fd730f230 scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents: 17675
diff changeset
723 else:
a41fd730f230 scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents: 17675
diff changeset
724 return self.base
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
725
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
726
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
727 opener: Type[vfs] = vfs
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
728
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
729
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
730 class proxyvfs(abstractvfs, abc.ABC):
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
731 def __init__(self, vfs: vfs) -> None:
17845
408ded42c5ec scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents: 17821
diff changeset
732 self.vfs = vfs
408ded42c5ec scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents: 17821
diff changeset
733
50365
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
734 @property
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
735 def createmode(self) -> Optional[int]:
50365
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
736 return self.vfs.createmode
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
737
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
738 def _auditpath(self, path: bytes, mode: bytes) -> None:
41097
adee334d94cd vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents: 41096
diff changeset
739 return self.vfs._auditpath(path, mode)
adee334d94cd vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents: 41096
diff changeset
740
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27851
diff changeset
741 @property
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
742 def options(self) -> Dict[bytes, Any]:
29727
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
743 return self.vfs.options
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
744
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
745 @options.setter
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
746 def options(self, value: Dict[bytes, Any]) -> None:
29727
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
747 self.vfs.options = value
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
748
51618
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
749 @property
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
750 def audit(self):
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
751 return self.vfs.audit
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
752
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
753
41096
6498f0e03526 vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents: 41095
diff changeset
754 class filtervfs(proxyvfs, abstractvfs):
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
755 '''Wrapper vfs for filtering filenames with a function.'''
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
756
51931
992fcf6b2473 typing: add a handful more annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51930
diff changeset
757 def __init__(self, vfs: vfs, filter: Callable[[bytes], bytes]) -> None:
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 33282
diff changeset
758 proxyvfs.__init__(self, vfs)
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
759 self._filter = filter
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
760
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
761 # TODO: The return type should be BinaryIO
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
762 def __call__(self, path: bytes, *args, **kwargs) -> Any:
17846
f42cf30873dc scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents: 17845
diff changeset
763 return self.vfs(self._filter(path), *args, **kwargs)
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
764
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
765 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
766 if path:
24628
a0b47885a1c5 vfs: make it possible to pass multiple path elements to join
Matt Harbison <matt_harbison@yahoo.com>
parents: 24582
diff changeset
767 return self.vfs.join(self._filter(self.vfs.reljoin(path, *insidef)))
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
768 else:
17846
f42cf30873dc scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents: 17845
diff changeset
769 return self.vfs.join(path)
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
770
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
771
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
772 filteropener: Type[filtervfs] = filtervfs
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
773
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
774
41096
6498f0e03526 vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents: 41095
diff changeset
775 class readonlyvfs(proxyvfs):
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
776 '''Wrapper vfs preventing any writing.'''
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
777
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
778 def __init__(self, vfs: vfs) -> None:
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 33282
diff changeset
779 proxyvfs.__init__(self, vfs)
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
780
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
781 # TODO: The return type should be BinaryIO
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
782 def __call__(self, path: bytes, mode: bytes = b'rb', *args, **kw) -> Any:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
783 if mode not in (b'r', b'rb'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
784 raise error.Abort(_(b'this vfs is read only'))
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
785 return self.vfs(path, mode, *args, **kw)
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
786
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
787 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
26156
a112fffdb632 scmutil.readonlyvfs: implement join
Siddharth Agarwal <sid0@fb.com>
parents: 26098
diff changeset
788 return self.vfs.join(path, *insidef)
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
789
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
790
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
791 class closewrapbase(abc.ABC):
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
792 """Base class of wrapper, which hooks closing
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
793
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
794 Do not instantiate outside the vfs layer.
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
795 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
796
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
797 def __init__(self, fh) -> None:
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43532
diff changeset
798 object.__setattr__(self, '_origfh', fh)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
799
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
800 def __getattr__(self, attr: str) -> Any:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
801 return getattr(self._origfh, attr)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
802
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
803 def __setattr__(self, attr: str, value: Any) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
804 return setattr(self._origfh, attr, value)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
805
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
806 def __delattr__(self, attr: str) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
807 return delattr(self._origfh, attr)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
808
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
809 def __enter__(self: _Tclosewrapbase) -> _Tclosewrapbase:
40468
8d9f366b7f19 vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents: 39909
diff changeset
810 self._origfh.__enter__()
8d9f366b7f19 vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents: 39909
diff changeset
811 return self
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
812
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
813 @abc.abstractmethod
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
814 def __exit__(self, exc_type, exc_value, exc_tb) -> None:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
815 ...
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
816
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
817 @abc.abstractmethod
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
818 def close(self) -> None:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
819 ...
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
820
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
821
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
822 class delayclosedfile(closewrapbase):
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
823 """Proxy for a file object whose close is delayed.
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
824
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
825 Do not instantiate outside the vfs layer.
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
826 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
827
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
828 def __init__(self, fh, closer) -> None:
52668
5cc8deb96b48 pyupgrade: modernize calls to superclass methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 52293
diff changeset
829 super().__init__(fh)
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43532
diff changeset
830 object.__setattr__(self, '_closer', closer)
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
831
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
832 def __exit__(self, exc_type, exc_value, exc_tb) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
833 self._closer.close(self._origfh)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
834
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
835 def close(self) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
836 self._closer.close(self._origfh)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
837
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
838
49037
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48966
diff changeset
839 class backgroundfilecloser:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
840 """Coordinates background closing of file handles on multiple threads."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
841
51929
fa9e8a6521c1 typing: manually add type annotations to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51928
diff changeset
842 def __init__(self, ui: uimod.ui, expectedcount: int = -1) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
843 self._running = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
844 self._entered = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
845 self._threads = []
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
846 self._threadexception = None
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
847
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
848 # Only Windows/NTFS has slow file closing. So only enable by default
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
849 # on that platform. But allow to be enabled elsewhere for testing.
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34040
diff changeset
850 defaultenabled = pycompat.iswindows
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
851 enabled = ui.configbool(b'worker', b'backgroundclose', defaultenabled)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
852
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
853 if not enabled:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
854 return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
855
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
856 # There is overhead to starting and stopping the background threads.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
857 # Don't do background processing unless the file count is large enough
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
858 # to justify it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
859 minfilecount = ui.configint(b'worker', b'backgroundcloseminfilecount')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
860 # FUTURE dynamically start background threads after minfilecount closes.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
861 # (We don't currently have any callers that don't know their file count)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
862 if expectedcount > 0 and expectedcount < minfilecount:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
863 return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
864
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
865 maxqueue = ui.configint(b'worker', b'backgroundclosemaxqueue')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
866 threadcount = ui.configint(b'worker', b'backgroundclosethreadcount')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
867
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
868 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
869 b'starting %d threads for background file closing\n' % threadcount
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
870 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
871
37890
8fb9985382be pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35725
diff changeset
872 self._queue = pycompat.queue.Queue(maxsize=maxqueue)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
873 self._running = True
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
874
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
875 for i in range(threadcount):
43510
ffd632c224c3 vfs: another bytes-str confusion on thread name
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
876 t = threading.Thread(target=self._worker, name='backgroundcloser')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
877 self._threads.append(t)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
878 t.start()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
879
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
880 def __enter__(self: _Tbackgroundfilecloser) -> _Tbackgroundfilecloser:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
881 self._entered = True
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
882 return self
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
883
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
884 def __exit__(self, exc_type, exc_value, exc_tb) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
885 self._running = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
886
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
887 # Wait for threads to finish closing so open files don't linger for
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
888 # longer than lifetime of context manager.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
889 for t in self._threads:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
890 t.join()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
891
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
892 def _worker(self) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
893 """Main routine for worker thread."""
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
894 while True:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
895 try:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
896 fh = self._queue.get(block=True, timeout=0.100)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
897 # Need to catch or the thread will terminate and
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
898 # we could orphan file descriptors.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
899 try:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
900 fh.close()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
901 except Exception as e:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
902 # Stash so can re-raise from main thread later.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
903 self._threadexception = e
37890
8fb9985382be pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35725
diff changeset
904 except pycompat.queue.Empty:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
905 if not self._running:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
906 break
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
907
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
908 def close(self, fh) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
909 """Schedule a file for closing."""
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
910 if not self._entered:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
911 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43090
diff changeset
912 _(b'can only call close() when context manager active')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
913 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
914
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
915 # If a background thread encountered an exception, raise now so we fail
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
916 # fast. Otherwise, we may potentially go on for minutes until the error
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
917 # is acted on.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
918 if self._threadexception:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
919 e = self._threadexception
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
920 self._threadexception = None
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
921 raise e
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
922
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
923 # If we're not actively running, close synchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
924 if not self._running:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
925 fh.close()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
926 return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
927
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
928 self._queue.put(fh, block=True, timeout=None)
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
929
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
930
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
931 class checkambigatclosing(closewrapbase):
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
932 """Proxy for a file object, to avoid ambiguity of file stat
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
933
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
934 See also util.filestat for detail about "ambiguity of file stat".
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
935
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
936 This proxy is useful only if the target file is guarded by any
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
937 lock (e.g. repo.lock or repo.wlock)
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
938
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
939 Do not instantiate outside the vfs layer.
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
940 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
941
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
942 def __init__(self, fh) -> None:
52668
5cc8deb96b48 pyupgrade: modernize calls to superclass methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 52293
diff changeset
943 super().__init__(fh)
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43532
diff changeset
944 object.__setattr__(self, '_oldstat', util.filestat.frompath(fh.name))
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
945
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
946 def _checkambig(self) -> None:
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
947 oldstat = self._oldstat
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
948 if oldstat.stat:
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
949 _avoidambig(self._origfh.name, oldstat)
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
950
51928
ad83e4f9b40e typing: correct pytype mistakes in `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51927
diff changeset
951 def __exit__(self, exc_type, exc_value, exc_tb) -> None:
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
952 self._origfh.__exit__(exc_type, exc_value, exc_tb)
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
953 self._checkambig()
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
954
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
955 def close(self) -> None:
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
956 self._origfh.close()
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
957 self._checkambig()