Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/vfs.py @ 51927:38720073aa34

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
typing: run `merge-pyi` on `mercurial/vfs.py` The *.pyi file was generated with pytype 2023.11.21. There were a few things here that were wrong (e.g. `writelines()` takes an `Iterable[bytes]`, not `bytes`, or inexplicable errors like importing several of the vfs classes from this very module), and those changes have been dropped manually here.
author Matt Harbison <matt_harbison@yahoo.com>
date Fri, 20 Sep 2024 13:38:13 -0400
parents e59e1d8d29d2
children ad83e4f9b40e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
31227
0f31830fbfc4 vfs: extract 'vfs' class and related code to a new 'vfs' module (API)
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31045
diff changeset
1 # vfs.py - Mercurial 'vfs' classes
13962
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45957
diff changeset
3 # Copyright Olivia Mackall <olivia@selenic.com>
13962
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
4 #
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
8b252e826c68 add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
7
51901
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51739
diff changeset
8 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51739
diff changeset
9
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
10 import abc
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
11 import contextlib
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
12 import os
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
13 import shutil
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
14 import stat
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
15 import threading
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
16 import typing
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
17
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
18 from typing import (
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
19 Any,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
20 Iterable,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
21 Iterator,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
22 List,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
23 Optional,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
24 Tuple,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
25 Type,
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
26 TypeVar,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
27 )
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
28
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
29 from .i18n import _
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
30 from . import (
34040
d5b2beca16c0 python3: wrap all uses of <exception>.strerror with strtolocal
Augie Fackler <raf@durin42.com>
parents: 33647
diff changeset
31 encoding,
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
32 error,
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
33 pathutil,
30318
af7c60988f6e py3: make scmutil.rcpath() return bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30109
diff changeset
34 pycompat,
27482
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
35 util,
dde3da2246f1 scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27093
diff changeset
36 )
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents: 18678
diff changeset
37
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
38 if typing.TYPE_CHECKING:
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
39 _Tbackgroundfilecloser = TypeVar(
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
40 '_Tbackgroundfilecloser', bound='backgroundfilecloser'
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
41 )
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
42 _Tclosewrapbase = TypeVar('_Tclosewrapbase', bound='closewrapbase')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
43
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
44
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
45 def _avoidambig(path: bytes, oldstat) -> None:
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
46 """Avoid file stat ambiguity forcibly
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
47
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
48 This function causes copying ``path`` file, if it is owned by
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
49 another (see issue5418 and issue5584 for detail).
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
50 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
51
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
52 def checkandavoid():
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
53 newstat = util.filestat.frompath(path)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
54 # return whether file stat ambiguity is (already) avoided
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
55 return not newstat.isambig(oldstat) or newstat.avoidambig(path, oldstat)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
56
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
57 if not checkandavoid():
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
58 # simply copy to change owner of path to get privilege to
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
59 # advance mtime (see issue5418)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
60 util.rename(util.mktempcopy(path), path)
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
61 checkandavoid()
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
62
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
63
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
64 class abstractvfs(abc.ABC):
14089
d3f7e110c3c0 opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14068
diff changeset
65 """Abstract base class; cannot be instantiated"""
d3f7e110c3c0 opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14068
diff changeset
66
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
67 # default directory separator for vfs
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
68 #
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
69 # Other vfs code always use `/` and this works fine because python file API
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
70 # abstract the use of `/` and make it work transparently. For consistency
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
71 # vfs will always use `/` when joining. This avoids some confusion in
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
72 # encoded vfs (see issue6546)
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
73 _dir_sep: bytes = b'/'
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
74
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
75 # TODO: type return, which is util.posixfile wrapped by a proxy
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
76 @abc.abstractmethod
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
77 def __call__(self, path: bytes, mode: bytes = b'rb', **kwargs) -> Any:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
78 ...
43515
b16912f8c07c vfs: add a NotImplementedError implementation of __call__
Augie Fackler <augie@google.com>
parents: 43514
diff changeset
79
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
80 @abc.abstractmethod
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
81 def _auditpath(self, path: bytes, mode: bytes) -> Any:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
82 ...
41094
d9b6b9ed96d8 vfs: add a `_auditpath` to abstract vfs
Boris Feld <boris.feld@octobus.net>
parents: 41093
diff changeset
83
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
84 @abc.abstractmethod
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
85 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
86 ...
43516
8b0fa4de0064 vfs: add NotImplementedError version of join
Augie Fackler <augie@google.com>
parents: 43515
diff changeset
87
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
88 def tryread(self, path: bytes) -> bytes:
16479
fc04698fa778 opener: coding style, use triple quotes for doc string
Thomas Arendsen Hein <thomas@intevation.de>
parents: 16455
diff changeset
89 '''gracefully return an empty string for missing files'''
16455
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
90 try:
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
91 return self.read(path)
49314
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
92 except FileNotFoundError:
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
93 pass
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
94 return b""
16455
154219f3a6a4 opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents: 16436
diff changeset
95
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
96 def tryreadlines(self, path: bytes, mode: bytes = b'rb') -> Any:
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
97 '''gracefully return an empty array for missing files'''
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
98 try:
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
99 return self.readlines(path, mode=mode)
49314
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
100 except FileNotFoundError:
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
101 pass
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
102 return []
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
103
29730
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29727
diff changeset
104 @util.propertycache
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29727
diff changeset
105 def open(self):
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
106 """Open ``path`` file, which is relative to vfs root.
23370
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
107
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
108 Newly created directories are marked as "not to be indexed by
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
109 the content indexing service", if ``notindexed`` is specified
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
110 for "write" mode access.
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
111 """
29730
2dd8c225e94c vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29727
diff changeset
112 return self.__call__
19897
896a4568def7 vfs: add "open()" for newly added code paths which open files via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19896
diff changeset
113
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
114 def read(self, path: bytes) -> bytes:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
115 with self(path, b'rb') as fp:
14097
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
116 return fp.read()
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
117
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
118 def readlines(self, path: bytes, mode: bytes = b'rb') -> Any:
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
119 with self(path, mode=mode) as fp:
23368
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
120 return fp.readlines()
bf8c3172255c vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23259
diff changeset
121
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
122 def write(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
123 self, path: bytes, data: bytes, backgroundclose=False, **kwargs
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
124 ) -> int:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
125 with self(path, b'wb', backgroundclose=backgroundclose, **kwargs) as fp:
14167
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14138
diff changeset
126 return fp.write(data)
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14138
diff changeset
127
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
128 def writelines(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
129 self, path: bytes, data: bytes, mode: bytes = b'wb', notindexed=False
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
130 ) -> None:
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
131 with self(path, mode=mode, notindexed=notindexed) as fp:
23371
1df6519eb3ab vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23370
diff changeset
132 return fp.writelines(data)
1df6519eb3ab vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23370
diff changeset
133
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
134 def append(self, path: bytes, data: bytes) -> int:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
135 with self(path, b'ab') as fp:
14097
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
136 return fp.write(data)
ca3376f044f8 opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14090
diff changeset
137
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
138 def basename(self, path: bytes) -> bytes:
25770
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
139 """return base element of a path (as os.path.basename would do)
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
140
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
141 This exists to allow handling of strange encoding if needed."""
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
142 return os.path.basename(path)
39de2e9cc6f4 vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25739
diff changeset
143
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
144 def chmod(self, path: bytes, mode: int) -> None:
20086
f3df2612f3c3 vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20085
diff changeset
145 return os.chmod(self.join(path), mode)
f3df2612f3c3 vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20085
diff changeset
146
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
147 def dirname(self, path: bytes) -> bytes:
25772
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
148 """return dirname element of a path (as os.path.dirname would do)
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
149
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
150 This exists to allow handling of strange encoding if needed."""
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
151 return os.path.dirname(path)
5471965af5cb vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25770
diff changeset
152
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
153 def exists(self, path: Optional[bytes] = None) -> bool:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
154 return os.path.exists(self.join(path))
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
155
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
156 def fstat(self, fp) -> os.stat_result:
19899
8c3dcbbfb5de changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19897
diff changeset
157 return util.fstat(fp)
8c3dcbbfb5de changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19897
diff changeset
158
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
159 def isdir(self, path: Optional[bytes] = None) -> bool:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
160 return os.path.isdir(self.join(path))
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
161
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
162 def isfile(self, path: Optional[bytes] = None) -> bool:
20085
589d6bb5b18d vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20045
diff changeset
163 return os.path.isfile(self.join(path))
589d6bb5b18d vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20045
diff changeset
164
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
165 def islink(self, path: Optional[bytes] = None) -> bool:
18949
138978f20180 localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18948
diff changeset
166 return os.path.islink(self.join(path))
138978f20180 localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18948
diff changeset
167
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
168 def isfileorlink(self, path: Optional[bytes] = None) -> bool:
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
169 """return whether path is a regular file or a symlink
27571
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
170
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
171 Unlike isfile, this doesn't follow symlinks."""
27571
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
172 try:
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
173 st = self.lstat(path)
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
174 except OSError:
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
175 return False
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
176 mode = st.st_mode
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
177 return stat.S_ISREG(mode) or stat.S_ISLNK(mode)
6a6e78f84cc6 merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents: 26836
diff changeset
178
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
179 def _join(self, *paths: bytes) -> bytes:
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
180 root_idx = 0
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
181 for idx, p in enumerate(paths):
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
182 if os.path.isabs(p) or p.startswith(self._dir_sep):
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
183 root_idx = idx
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
184 if root_idx != 0:
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
185 paths = paths[root_idx:]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
186 paths = [p for p in paths if p]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
187 return self._dir_sep.join(paths)
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
188
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
189 def reljoin(self, *paths: bytes) -> bytes:
23581
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
190 """join various elements of a path together (as os.path.join would do)
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
191
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
192 The vfs base is not injected so that path stay relative. This exists
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
193 to allow handling of strange encoding if needed."""
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
194 return self._join(*paths)
23581
aed981c7bebf vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23539
diff changeset
195
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
196 def split(self, path: bytes) -> Tuple[bytes, bytes]:
23582
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
197 """split top-most element of a path (as os.path.split would do)
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
198
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
199 This exists to allow handling of strange encoding if needed."""
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
200 return os.path.split(path)
7559dc8c4238 vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23581
diff changeset
201
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
202 def lexists(self, path: Optional[bytes] = None) -> bool:
21563
764b691b8bda vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21111
diff changeset
203 return os.path.lexists(self.join(path))
764b691b8bda vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21111
diff changeset
204
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
205 def lstat(self, path: Optional[bytes] = None) -> os.stat_result:
19900
7c21e3398931 context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19899
diff changeset
206 return os.lstat(self.join(path))
7c21e3398931 context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19899
diff changeset
207
51649
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
208 def is_mmap_safe(self, path: Optional[bytes] = None) -> bool:
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
209 """return True if it is safe to read a file content as mmap
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
210
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
211 This focus on the file system aspect of such safety, the application
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
212 logic around that file is not taken into account, so caller need to
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
213 make sure the file won't be truncated in a way that will create SIGBUS
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
214 on access.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
215
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
216
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
217 The initial motivation for this logic is that if mmap is used on NFS
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
218 and somebody deletes the mapped file (e.g. by renaming on top of it),
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
219 then you get SIGBUS, which can be pretty disruptive: we get core dump
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
220 reports, and the process terminates without writing to the blackbox.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
221
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
222 Instead, in this situation we prefer to read the file normally.
51649
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
223 The risk of ESTALE in the middle of the read remains, but it's
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
224 smaller because we read sooner and the error should be reported
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
225 just as any other error.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
226
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
227 Note that python standard library does not offer the necessary function
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
228 to detect the file stem bits. So this detection rely on compiled bits
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
229 and is not available in pure python.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
230 """
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
231 # XXX Since we already assume a vfs to address a consistent file system
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
232 # in other location, we could determine the fstype once for the root
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
233 # and cache that value.
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
234 fstype = util.getfstype(self.join(path))
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
235 return fstype is not None and fstype != b'nfs'
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51618
diff changeset
236
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
237 def listdir(self, path: Optional[bytes] = None) -> List[bytes]:
21799
dfacdd6a111e vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21716
diff changeset
238 return os.listdir(self.join(path))
dfacdd6a111e vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21716
diff changeset
239
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
240 def makedir(self, path: Optional[bytes] = None, notindexed=True) -> None:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
241 return util.makedir(self.join(path), notindexed)
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
242
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
243 def makedirs(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
244 self, path: Optional[bytes] = None, mode: Optional[int] = None
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
245 ) -> None:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
246 return util.makedirs(self.join(path), mode)
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
247
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
248 def makelock(self, info, path: bytes) -> None:
20090
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
249 return util.makelock(info, self.join(path))
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
250
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
251 def mkdir(self, path: Optional[bytes] = None) -> None:
17723
ab23768746fd scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17681
diff changeset
252 return os.mkdir(self.join(path))
ab23768746fd scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17681
diff changeset
253
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
254 def mkstemp(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
255 self,
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
256 suffix: bytes = b'',
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
257 prefix: bytes = b'tmp',
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
258 dir: Optional[bytes] = None,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
259 ) -> Tuple[int, bytes]:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
260 fd, name = pycompat.mkstemp(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
261 suffix=suffix, prefix=prefix, dir=self.join(dir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
262 )
20980
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
263 dname, fname = util.split(name)
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
264 if dir:
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
265 return fd, os.path.join(dir, fname)
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
266 else:
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
267 return fd, fname
6fb4c94ae300 vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20862
diff changeset
268
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
269 def readdir(
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
270 self, path: Optional[bytes] = None, stat=None, skip=None
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
271 ) -> Any:
32248
d74b0cff94a9 osutil: proxy through util (and platform) modules (API)
Yuya Nishihara <yuya@tcha.org>
parents: 31649
diff changeset
272 return util.listdir(self.join(path), stat, skip)
17747
aad3bce98f76 store: invoke "osutil.listdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17726
diff changeset
273
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
274 def readlock(self, path: bytes) -> bytes:
20090
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
275 return util.readlock(self.join(path))
88d8e568add1 vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20086
diff changeset
276
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
277 def rename(self, src: bytes, dst: bytes, checkambig=False) -> None:
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
278 """Rename from src to dst
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
279
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
280 checkambig argument is used with util.filestat, and is useful
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
281 only if destination file is guarded by any lock
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
282 (e.g. repo.lock or repo.wlock).
33282
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
283
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
284 To avoid file stat ambiguity forcibly, checkambig=True involves
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
285 copying ``src`` file, if it is owned by another. Therefore, use
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
286 checkambig=True only in limited cases (see also issue5418 and
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
287 issue5584 for detail).
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
288 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
289 self._auditpath(dst, b'w')
32768
ed66ec39933f vfs: create copy at renaming to avoid file stat ambiguity if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32767
diff changeset
290 srcpath = self.join(src)
29203
731ced087a4b vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29202
diff changeset
291 dstpath = self.join(dst)
51922
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
292 oldstat = util.filestat.frompath(dstpath) if checkambig else None
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
293
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
294 util.rename(srcpath, dstpath)
adbb183c2f27 vfs: simplify the `abstractvfs.rename()` implementation
Matt Harbison <matt_harbison@yahoo.com>
parents: 51921
diff changeset
295
29203
731ced087a4b vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29202
diff changeset
296 if oldstat and oldstat.stat:
33281
6af0f023d014 vfs: replace avoiding ambiguity in abstractvfs.rename with _avoidambig
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33280
diff changeset
297 _avoidambig(dstpath, oldstat)
18948
2f05fa162316 localrepo: use "vfs.rename()" instead of "util.rename()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18945
diff changeset
298
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
299 def readlink(self, path: bytes) -> bytes:
39909
5fe0b880200e py3: convert os.readlink() path to native strings on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39481
diff changeset
300 return util.readlink(self.join(path))
18950
647e3b0c8751 localrepo: use "vfs.readlink()" instead of "os.readlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18949
diff changeset
301
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
302 def removedirs(self, path: Optional[bytes] = None) -> None:
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
303 """Remove a leaf directory and all empty intermediate ones"""
24693
0d28b0df77ea vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24689
diff changeset
304 return util.removedirs(self.join(path))
0d28b0df77ea vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24689
diff changeset
305
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
306 def rmdir(self, path: Optional[bytes] = None) -> None:
39481
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38497
diff changeset
307 """Remove an empty directory."""
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38497
diff changeset
308 return os.rmdir(self.join(path))
3dd34b401bc2 merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38497
diff changeset
309
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
310 def rmtree(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
311 self, path: Optional[bytes] = None, ignore_errors=False, forcibly=False
51739
be6d8ea6d3d2 typing: add a trivial type hint to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51649
diff changeset
312 ) -> None:
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
313 """Remove a directory tree recursively
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
314
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
315 If ``forcibly``, this tries to remove READ-ONLY files, too.
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
316 """
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
317 if forcibly:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
318
50750
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
319 def onexc(function, path, excinfo):
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
320 if function is not os.remove:
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
321 raise
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
322 # read-only files cannot be unlinked under Windows
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
323 s = os.stat(path)
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
324 if (s.st_mode & stat.S_IWRITE) != 0:
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
325 raise
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
326 os.chmod(path, stat.S_IMODE(s.st_mode) | stat.S_IWRITE)
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
327 os.remove(path)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
328
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
329 else:
50750
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
330 onexc = None
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
331 try:
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
332 # pytype: disable=wrong-keyword-args
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
333 return shutil.rmtree(
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
334 self.join(path), ignore_errors=ignore_errors, onexc=onexc
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
335 )
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
336 # pytype: enable=wrong-keyword-args
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
337 except TypeError: # onexc was introduced in Python 3.12
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
338 return shutil.rmtree(
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
339 self.join(path), ignore_errors=ignore_errors, onerror=onexc
f173c2c23289 vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents: 50365
diff changeset
340 )
24689
ca3a90096c95 vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24635
diff changeset
341
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
342 def setflags(self, path: bytes, l: bool, x: bool) -> None:
18951
d13916a00b7e localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18950
diff changeset
343 return util.setflags(self.join(path), l, x)
d13916a00b7e localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18950
diff changeset
344
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
345 def stat(self, path: Optional[bytes] = None) -> os.stat_result:
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
346 return os.stat(self.join(path))
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
347
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
348 def unlink(self, path: Optional[bytes] = None) -> None:
19895
37c0d93fb166 bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19655
diff changeset
349 return util.unlink(self.join(path))
37c0d93fb166 bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19655
diff changeset
350
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
351 def tryunlink(self, path: Optional[bytes] = None) -> bool:
31548
fad440db3565 vfs: add tryunlink method
Ryan McElroy <rmcelroy@fb.com>
parents: 31318
diff changeset
352 """Attempt to remove a file, ignoring missing file errors."""
51428
187c5769a629 vfs: have tryunlink tell what it did
Georges Racinet <georges.racinet@octobus.net>
parents: 50952
diff changeset
353 return util.tryunlink(self.join(path))
31548
fad440db3565 vfs: add tryunlink method
Ryan McElroy <rmcelroy@fb.com>
parents: 31318
diff changeset
354
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
355 def unlinkpath(
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
356 self, path: Optional[bytes] = None, ignoremissing=False, rmdir=True
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
357 ) -> None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
358 return util.unlinkpath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
359 self.join(path), ignoremissing=ignoremissing, rmdir=rmdir
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
360 )
21716
90f9be5adade vfs: add unlinkpath to vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents: 21563
diff changeset
361
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
362 def utime(self, path: Optional[bytes] = None, t=None) -> None:
19896
af03279c766a bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19895
diff changeset
363 return os.utime(self.join(path), t)
af03279c766a bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19895
diff changeset
364
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
365 def walk(self, path: Optional[bytes] = None, onerror=None):
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
366 """Yield (dirpath, dirs, files) tuple for each directory under path
24725
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
367
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
368 ``dirpath`` is relative one from the root of this vfs. This
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
369 uses ``os.sep`` as path separator, even you specify POSIX
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
370 style ``path``.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
371
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
372 "The root of this vfs" is represented as empty ``dirpath``.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
373 """
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
374 root = os.path.normpath(self.join(None))
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
375 # when dirpath == root, dirpath[prefixlen:] becomes empty
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
376 # because len(dirpath) < prefixlen.
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
377 prefixlen = len(pathutil.normasprefix(root))
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
378 for dirpath, dirs, files in os.walk(self.join(path), onerror=onerror):
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
379 yield (dirpath[prefixlen:], dirs, files)
ee751d47cf2c vfs: add walk
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24723
diff changeset
380
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
381 @contextlib.contextmanager
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
382 def backgroundclosing(self, ui, expectedcount=-1):
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
383 """Allow files to be closed asynchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
384
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
385 When this context manager is active, ``backgroundclose`` can be passed
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
386 to ``__call__``/``open`` to result in the file possibly being closed
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
387 asynchronously, on a background thread.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
388 """
35432
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
389 # Sharing backgroundfilecloser between threads is complex and using
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
390 # multiple instances puts us at risk of running out of file descriptors
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
391 # only allow to use backgroundfilecloser when in main thread.
51920
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
392 if threading.current_thread() is not threading.main_thread():
35432
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
393 yield
60f2a215faa7 workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents: 34645
diff changeset
394 return
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
395 vfs = getattr(self, 'vfs', self)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
396 if getattr(vfs, '_backgroundfilecloser', None):
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29373
diff changeset
397 raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
398 _(b'can only have 1 active background file closer')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
399 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
400
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
401 with backgroundfilecloser(ui, expectedcount=expectedcount) as bfc:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
402 try:
43530
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
403 vfs._backgroundfilecloser = (
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
404 bfc # pytype: disable=attribute-error
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
405 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
406 yield bfc
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
407 finally:
43530
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
408 vfs._backgroundfilecloser = (
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
409 None # pytype: disable=attribute-error
fc19f8ab8199 vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents: 43516
diff changeset
410 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
411
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
412 def register_file(self, path) -> None:
47449
9ab54aa56982 vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47192
diff changeset
413 """generic hook point to lets fncache steer its stew"""
9ab54aa56982 vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47192
diff changeset
414
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
415
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
416 class vfs(abstractvfs):
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
417 """Operate files relative to a base directory
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
418
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
419 This class is used to hide the details of COW semantics and
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
420 remote file access from higher level code.
33647
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
421
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
422 'cacheaudited' should be enabled only if (a) vfs object is short-lived, or
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
423 (b) the base directory is managed by hg and considered sort-of append-only.
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
424 See pathutil.pathauditor() for details.
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
425 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
426
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
427 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
428 self,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
429 base: bytes,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
430 audit=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
431 cacheaudited=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
432 expandpath=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
433 realpath=False,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
434 ) -> None:
18945
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
435 if expandpath:
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
436 base = util.expandpath(base)
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
437 if realpath:
e75b72fffdfe vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18900
diff changeset
438 base = os.path.realpath(base)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
439 self.base = base
33259
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
440 self._audit = audit
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
441 if audit:
33647
377e8ddaebef pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents: 33435
diff changeset
442 self.audit = pathutil.pathauditor(self.base, cached=cacheaudited)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
443 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
444 self.audit = lambda path, mode=None: True
33259
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
445 self.createmode = None
6fb5c5096887 vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33257
diff changeset
446 self._trustnlink = None
43027
3518da504303 vfs: give all vfs an options attribute by default
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41098
diff changeset
447 self.options = {}
17554
5450c8ad9d98 scmutil: turn opener._audit into a property, mustaudit
Bryan O'Sullivan <bryano@fb.com>
parents: 17248
diff changeset
448
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
449 @util.propertycache
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
450 def _cansymlink(self) -> bool:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
451 return util.checklink(self.base)
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
452
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
453 @util.propertycache
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
454 def _chmod(self):
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
455 return util.checkexec(self.base)
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
456
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
457 def _fixfilemode(self, name) -> None:
18192
f9a89bdd64a6 scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents: 17850
diff changeset
458 if self.createmode is None or not self._chmod:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
459 return
25658
e93036747902 global: mass rewrite to use modern octal syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25629
diff changeset
460 os.chmod(name, self.createmode & 0o666)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
461
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
462 def _auditpath(self, path, mode) -> None:
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
463 if self._audit:
41093
c8006a25b845 vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents: 40999
diff changeset
464 if os.path.isabs(path) and path.startswith(self.base):
c8006a25b845 vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents: 40999
diff changeset
465 path = os.path.relpath(path, self.base)
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
466 r = util.checkosfilename(path)
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
467 if r:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
468 raise error.Abort(b"%s: %r" % (r, path))
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
469 self.audit(path, mode=mode)
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
470
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
471 def isfileorlink_checkdir(
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
472 self, dircache, path: Optional[bytes] = None
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
473 ) -> bool:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
474 """return True if the path is a regular file or a symlink and
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
475 the directories along the path are "normal", that is
49985
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
476 not symlinks or nested hg repositories.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
477
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
478 Ignores the `_audit` setting, and checks the directories regardless.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
479 `dircache` is used to cache the directory checks.
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
480 """
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
481 try:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
482 for prefix in pathutil.finddirs_rev_noroot(util.localpath(path)):
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
483 if prefix in dircache:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
484 res = dircache[prefix]
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
485 else:
49985
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
486 res = pathutil.pathauditor._checkfs_exists(
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
487 self.base, prefix, path
bc83ebe07bf0 pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49982
diff changeset
488 )
49982
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
489 dircache[prefix] = res
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
490 if not res:
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
491 return False
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
492 except (OSError, error.Abort):
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
493 return False
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
494 return self.isfileorlink(path)
b7cf91ef03ba merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 49723
diff changeset
495
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
496 def __call__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
497 self,
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
498 path: bytes,
49723
2506c3ac73f4 vfs: make the default opener mode binary
Matt Harbison <matt_harbison@yahoo.com>
parents: 49722
diff changeset
499 mode: bytes = b"rb",
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
500 atomictemp=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
501 notindexed=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
502 backgroundclose=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
503 checkambig=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
504 auditpath=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
505 makeparentdirs=True,
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
506 ) -> Any:
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
507 """Open ``path`` file, which is relative to vfs root.
23370
46265d0f0c7b vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23368
diff changeset
508
40825
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
509 By default, parent directories are created as needed. Newly created
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
510 directories are marked as "not to be indexed by the content indexing
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
511 service", if ``notindexed`` is specified for "write" mode access.
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
512 Set ``makeparentdirs=False`` to not create directories implicitly.
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
513
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
514 If ``backgroundclose`` is passed, the file may be closed asynchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
515 It can only be used if the ``self.backgroundclosing()`` context manager
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
516 is active. This should only be specified if the following criteria hold:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
517
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
518 1. There is a potential for writing thousands of files. Unless you
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
519 are writing thousands of files, the performance benefits of
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
520 asynchronously closing files is not realized.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
521 2. Files are opened exactly once for the ``backgroundclosing``
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
522 active duration and are therefore free of race conditions between
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
523 closing a file on a background thread and reopening it. (If the
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
524 file were opened multiple times, there could be unflushed data
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
525 because the original file handle hasn't been flushed/closed yet.)
29202
76f1ea360c7e vfs: make atomictempfile avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29119
diff changeset
526
44575
77d48738b8e0 vfs: fix typo in comment (remove extra "l")
Kyle Lippincott <spectral@google.com>
parents: 43554
diff changeset
527 ``checkambig`` argument is passed to atomictempfile (valid
29367
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
528 only for writing), and is useful only if target file is
4e6e280e238f doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29341
diff changeset
529 guarded by any lock (e.g. repo.lock or repo.wlock).
33282
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
530
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
531 To avoid file stat ambiguity forcibly, checkambig=True involves
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
532 copying ``path`` file opened in "append" mode (e.g. for
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
533 truncation), if it is owned by another. Therefore, use
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
534 combination of append mode and checkambig=True only in limited
d1db7af81548 vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33281
diff changeset
535 cases (see also issue5418 and issue5584 for detail).
45957
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44575
diff changeset
536 """
33257
15e9cbe6ae49 vfs: simplify path audit disabling in stream clone
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33234
diff changeset
537 if auditpath:
40788
34f15db81cf0 vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents: 39909
diff changeset
538 self._auditpath(path, mode)
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
539 f = self.join(path)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
540
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
541 if b"b" not in mode:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
542 mode += b"b" # for that other OS
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
543
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
544 nlink = -1
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
545 if mode not in (b'r', b'rb'):
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
546 dirname, basename = util.split(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
547 # If basename is empty, then the path is malformed because it points
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
548 # to a directory. Let the posixfile() call below raise IOError.
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
549 if basename:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
550 if atomictemp:
40825
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
551 if makeparentdirs:
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
552 util.makedirs(dirname, self.createmode, notindexed)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
553 return util.atomictempfile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
554 f, mode, self.createmode, checkambig=checkambig
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
555 )
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
556 try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
557 if b'w' in mode:
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
558 util.unlink(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
559 nlink = 0
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
560 else:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
561 # nlinks() may behave differently for files on Windows
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
562 # shares if the file is open.
27706
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
563 with util.posixfile(f):
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
564 nlink = util.nlinks(f)
22e362da27cf scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27651
diff changeset
565 if nlink < 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
566 nlink = 2 # force mktempcopy (issue1922)
49314
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49037
diff changeset
567 except FileNotFoundError:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
568 nlink = 0
40825
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
569 if makeparentdirs:
03bca908d9fb vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents: 40788
diff changeset
570 util.makedirs(dirname, self.createmode, notindexed)
17937
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
571 if nlink > 0:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
572 if self._trustnlink is None:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
573 self._trustnlink = nlink > 1 or util.checknlink(f)
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
574 if nlink > 1 or not self._trustnlink:
3cb032d50447 vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents: 17850
diff changeset
575 util.rename(util.mktempcopy(f), f)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
576 fp = util.posixfile(f, mode)
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
577 if nlink == 0:
17763
13070de77c86 vfs: backout fchmod change from 76b73ce0ffac
Matt Mackall <mpm@selenic.com>
parents: 17752
diff changeset
578 self._fixfilemode(f)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
579
30000
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29999
diff changeset
580 if checkambig:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
581 if mode in (b'r', b'rb'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
582 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
583 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
584 b'implementation error: mode %s is not'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
585 b' valid for checkambig=True'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
586 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
587 % mode
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
588 )
30000
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29999
diff changeset
589 fp = checkambigatclosing(fp)
9766d88c2465 vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29999
diff changeset
590
51920
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
591 if (
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
592 backgroundclose
1edac12af730 vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents: 51901
diff changeset
593 and threading.current_thread() is threading.main_thread()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
594 ):
43532
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
595 if (
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
596 not self._backgroundfilecloser # pytype: disable=attribute-error
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
597 ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
598 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
599 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
600 b'backgroundclose can only be used when a '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
601 b'backgroundclosing context manager is active'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
602 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
603 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
604
43532
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
605 fp = delayclosedfile(
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
606 fp,
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
607 self._backgroundfilecloser, # pytype: disable=attribute-error
da9ecbb10368 vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents: 43530
diff changeset
608 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
609
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
610 return fp
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
611
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
612 def symlink(self, src: bytes, dst: bytes) -> None:
18327
4aecdb91443c scmutil: simplify vfs.audit - drop wrapped vfs.auditor
Mads Kiilerich <mads@kiilerich.com>
parents: 18316
diff changeset
613 self.audit(dst)
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
614 linkname = self.join(dst)
31555
18b9d9b95719 vfs: use tryunlink
Ryan McElroy <rmcelroy@fb.com>
parents: 31548
diff changeset
615 util.tryunlink(linkname)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
616
29017
07be86828e79 util: fix race in makedirs()
Adam Simpkins <simpkins@fb.com>
parents: 28819
diff changeset
617 util.makedirs(os.path.dirname(linkname), self.createmode)
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
618
14261
e3649bcca3f6 opener: rename _can_symlink to _cansymlink
Adrian Buehlmann <adrian@cadifra.com>
parents: 14236
diff changeset
619 if self._cansymlink:
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
620 try:
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
621 os.symlink(src, linkname)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25658
diff changeset
622 except OSError as err:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
623 raise OSError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
624 err.errno,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
625 _(b'could not symlink to %r: %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
626 % (src, encoding.strtolocal(err.strerror)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
627 linkname,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
628 )
13970
d13913355390 move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13962
diff changeset
629 else:
17768
9837cafc25b1 vfs: use self.write to write symlink placeholders
Matt Mackall <mpm@selenic.com>
parents: 17763
diff changeset
630 self.write(dst, src)
13971
bfeaa88b875d move canonpath from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents: 13970
diff changeset
631
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
632 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
17161
be016e96117a localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17157
diff changeset
633 if path:
47804
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
634 parts = [self.base, path]
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
635 parts.extend(insidef)
98c3fa6a3ac2 vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47449
diff changeset
636 return self._join(*parts)
17681
a41fd730f230 scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents: 17675
diff changeset
637 else:
a41fd730f230 scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents: 17675
diff changeset
638 return self.base
16199
8181bd808dc5 scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents: 16198
diff changeset
639
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
640
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
641 opener: Type[vfs] = vfs
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
642
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
643
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
644 class proxyvfs(abstractvfs, abc.ABC):
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
645 def __init__(self, vfs: "vfs"):
17845
408ded42c5ec scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents: 17821
diff changeset
646 self.vfs = vfs
408ded42c5ec scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents: 17821
diff changeset
647
50365
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
648 @property
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
649 def createmode(self):
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
650 return self.vfs.createmode
d1d458fb96a5 vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49985
diff changeset
651
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
652 def _auditpath(self, path, mode) -> None:
41097
adee334d94cd vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents: 41096
diff changeset
653 return self.vfs._auditpath(path, mode)
adee334d94cd vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents: 41096
diff changeset
654
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27851
diff changeset
655 @property
29727
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
656 def options(self):
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
657 return self.vfs.options
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
658
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
659 @options.setter
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
660 def options(self, value):
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
661 self.vfs.options = value
69109052d9ac auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents: 29417
diff changeset
662
51618
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
663 @property
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
664 def audit(self):
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
665 return self.vfs.audit
b5500857e173 proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51428
diff changeset
666
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
667
41096
6498f0e03526 vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents: 41095
diff changeset
668 class filtervfs(proxyvfs, abstractvfs):
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
669 '''Wrapper vfs for filtering filenames with a function.'''
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
670
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
671 def __init__(self, vfs: "vfs", filter):
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 33282
diff changeset
672 proxyvfs.__init__(self, vfs)
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
673 self._filter = filter
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
674
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
675 def __call__(self, path: bytes, *args, **kwargs) -> Any:
17846
f42cf30873dc scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents: 17845
diff changeset
676 return self.vfs(self._filter(path), *args, **kwargs)
14090
e24b5e3c2f27 add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14089
diff changeset
677
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
678 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
679 if path:
24628
a0b47885a1c5 vfs: make it possible to pass multiple path elements to join
Matt Harbison <matt_harbison@yahoo.com>
parents: 24582
diff changeset
680 return self.vfs.join(self._filter(self.vfs.reljoin(path, *insidef)))
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
681 else:
17846
f42cf30873dc scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents: 17845
diff changeset
682 return self.vfs.join(path)
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17723
diff changeset
683
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
684
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
685 filteropener: Type[filtervfs] = filtervfs
17649
f65c6a5f256c scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17561
diff changeset
686
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
687
41096
6498f0e03526 vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents: 41095
diff changeset
688 class readonlyvfs(proxyvfs):
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
689 '''Wrapper vfs preventing any writing.'''
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
690
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
691 def __init__(self, vfs: "vfs"):
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 33282
diff changeset
692 proxyvfs.__init__(self, vfs)
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
693
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
694 def __call__(self, path: bytes, mode: bytes = b'rb', *args, **kw) -> Any:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
695 if mode not in (b'r', b'rb'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
696 raise error.Abort(_(b'this vfs is read only'))
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
697 return self.vfs(path, mode, *args, **kw)
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
698
49722
cc9a60050a07 typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 49314
diff changeset
699 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
26156
a112fffdb632 scmutil.readonlyvfs: implement join
Siddharth Agarwal <sid0@fb.com>
parents: 26098
diff changeset
700 return self.vfs.join(path, *insidef)
18213
c38a62af000e vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18206
diff changeset
701
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
702
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
703 class closewrapbase(abc.ABC):
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
704 """Base class of wrapper, which hooks closing
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
705
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
706 Do not instantiate outside the vfs layer.
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
707 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
708
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
709 def __init__(self, fh) -> None:
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43532
diff changeset
710 object.__setattr__(self, '_origfh', fh)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
711
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
712 def __getattr__(self, attr) -> Any:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
713 return getattr(self._origfh, attr)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
714
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
715 def __setattr__(self, attr, value) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
716 return setattr(self._origfh, attr, value)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
717
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
718 def __delattr__(self, attr) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
719 return delattr(self._origfh, attr)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
720
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
721 def __enter__(self: _Tclosewrapbase) -> _Tclosewrapbase:
40468
8d9f366b7f19 vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents: 39909
diff changeset
722 self._origfh.__enter__()
8d9f366b7f19 vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents: 39909
diff changeset
723 return self
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
724
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
725 @abc.abstractmethod
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
726 def __exit__(self, exc_type, exc_value, exc_tb):
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
727 ...
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
728
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
729 @abc.abstractmethod
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
730 def close(self):
51921
f79f98733a5b vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents: 51920
diff changeset
731 ...
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
732
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
733
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
734 class delayclosedfile(closewrapbase):
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
735 """Proxy for a file object whose close is delayed.
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
736
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
737 Do not instantiate outside the vfs layer.
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
738 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
739
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
740 def __init__(self, fh, closer) -> None:
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
741 super(delayclosedfile, self).__init__(fh)
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43532
diff changeset
742 object.__setattr__(self, '_closer', closer)
29998
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
743
0c40e64d6154 scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29813
diff changeset
744 def __exit__(self, exc_type, exc_value, exc_tb):
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
745 self._closer.close(self._origfh)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
746
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
747 def close(self) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
748 self._closer.close(self._origfh)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
749
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
750
49037
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48966
diff changeset
751 class backgroundfilecloser:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
752 """Coordinates background closing of file handles on multiple threads."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
753
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
754 def __init__(self, ui, expectedcount=-1) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
755 self._running = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
756 self._entered = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
757 self._threads = []
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
758 self._threadexception = None
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
759
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
760 # Only Windows/NTFS has slow file closing. So only enable by default
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
761 # on that platform. But allow to be enabled elsewhere for testing.
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34040
diff changeset
762 defaultenabled = pycompat.iswindows
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
763 enabled = ui.configbool(b'worker', b'backgroundclose', defaultenabled)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
764
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
765 if not enabled:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
766 return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
767
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
768 # There is overhead to starting and stopping the background threads.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
769 # Don't do background processing unless the file count is large enough
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
770 # to justify it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
771 minfilecount = ui.configint(b'worker', b'backgroundcloseminfilecount')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
772 # FUTURE dynamically start background threads after minfilecount closes.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
773 # (We don't currently have any callers that don't know their file count)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
774 if expectedcount > 0 and expectedcount < minfilecount:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
775 return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
776
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
777 maxqueue = ui.configint(b'worker', b'backgroundclosemaxqueue')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
778 threadcount = ui.configint(b'worker', b'backgroundclosethreadcount')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
779
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
780 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
781 b'starting %d threads for background file closing\n' % threadcount
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
782 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
783
37890
8fb9985382be pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35725
diff changeset
784 self._queue = pycompat.queue.Queue(maxsize=maxqueue)
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
785 self._running = True
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
786
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
787 for i in range(threadcount):
43510
ffd632c224c3 vfs: another bytes-str confusion on thread name
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
788 t = threading.Thread(target=self._worker, name='backgroundcloser')
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
789 self._threads.append(t)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
790 t.start()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
791
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
792 def __enter__(self: _Tbackgroundfilecloser) -> _Tbackgroundfilecloser:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
793 self._entered = True
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
794 return self
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
795
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
796 def __exit__(self, exc_type, exc_value, exc_tb) -> None: # TODO
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
797 self._running = False
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
798
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
799 # Wait for threads to finish closing so open files don't linger for
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
800 # longer than lifetime of context manager.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
801 for t in self._threads:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
802 t.join()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
803
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
804 def _worker(self) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
805 """Main routine for worker thread."""
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
806 while True:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
807 try:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
808 fh = self._queue.get(block=True, timeout=0.100)
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
809 # Need to catch or the thread will terminate and
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
810 # we could orphan file descriptors.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
811 try:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
812 fh.close()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
813 except Exception as e:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
814 # Stash so can re-raise from main thread later.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
815 self._threadexception = e
37890
8fb9985382be pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35725
diff changeset
816 except pycompat.queue.Empty:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
817 if not self._running:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
818 break
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
819
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
820 def close(self, fh) -> None:
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
821 """Schedule a file for closing."""
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
822 if not self._entered:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
823 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43090
diff changeset
824 _(b'can only call close() when context manager active')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
825 )
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
826
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
827 # If a background thread encountered an exception, raise now so we fail
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
828 # fast. Otherwise, we may potentially go on for minutes until the error
27895
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
829 # is acted on.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
830 if self._threadexception:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
831 e = self._threadexception
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
832 self._threadexception = None
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
833 raise e
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
834
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
835 # If we're not actively running, close synchronously.
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
836 if not self._running:
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
837 fh.close()
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
838 return
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
839
2d6a89e79b48 scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27879
diff changeset
840 self._queue.put(fh, block=True, timeout=None)
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
841
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
842
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
843 class checkambigatclosing(closewrapbase):
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
844 """Proxy for a file object, to avoid ambiguity of file stat
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
845
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
846 See also util.filestat for detail about "ambiguity of file stat".
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
847
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
848 This proxy is useful only if the target file is guarded by any
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
849 lock (e.g. repo.lock or repo.wlock)
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
850
51923
e59e1d8d29d2 vfs: do minor copyediting on comments and doc strings
Matt Harbison <matt_harbison@yahoo.com>
parents: 51922
diff changeset
851 Do not instantiate outside the vfs layer.
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
852 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43027
diff changeset
853
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
854 def __init__(self, fh) -> None:
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
855 super(checkambigatclosing, self).__init__(fh)
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43532
diff changeset
856 object.__setattr__(self, '_oldstat', util.filestat.frompath(fh.name))
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
857
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
858 def _checkambig(self) -> None:
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
859 oldstat = self._oldstat
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
860 if oldstat.stat:
33280
646352291f5b vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 33259
diff changeset
861 _avoidambig(self._origfh.name, oldstat)
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
862
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
863 def __exit__(self, exc_type, exc_value, exc_tb) -> None: # TODO
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
864 self._origfh.__exit__(exc_type, exc_value, exc_tb)
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
865 self._checkambig()
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
866
51927
38720073aa34 typing: run `merge-pyi` on `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents: 51923
diff changeset
867 def close(self) -> None:
29999
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
868 self._origfh.close()
57830bd0e787 scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29998
diff changeset
869 self._checkambig()