Mercurial > public > mercurial-scm > hg-stable
annotate mercurial/vfs.py @ 51921:f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
The latter confuses PyCharm after adding more type annotations when, for
example, `abstractvfs.rename()` calls `_auditpath()`- the latter unconditionally
raised an error, so PyCharm thought the code that came after is unreachable. It
also tricked pytype into marking the return type as `Never`, which isn't
available until Python 3.11 (outside of `typing_extensions`).
This also avoid PyCharm warnings that the call to the superclass constructor was
missed (it couldn't be called because it raised an error to prevent
instantiation).
The statichttprepo module needed to be given an override for one of the abstract
methods, so that it can be instantiated. In `abstractvfs`, this method is only
called by `rename()`, so I think we can leave this empty. We raise an error in
case somebody accidentally calls it in the future- it would have raised this
same error prior to this change.
I couldn't wrangle `import-checker.py` into accepting importing `ABC` and
`abstractmethod`- for each subsequent import, it reports something like:
stdlib import "contextlib" follows local import: abc
I suspect the problem is that near the `if fullname != '__future__'` check, if
the module doesn't fall into the error case, `seenlocal` gets set to the module
name. That causes it to be treated like a local module on the next iteration,
even though it is in `stdlib_modules`.
author | Matt Harbison <matt_harbison@yahoo.com> |
---|---|
date | Fri, 20 Sep 2024 00:07:39 -0400 |
parents | 1edac12af730 |
children | adbb183c2f27 |
rev | line source |
---|---|
31227
0f31830fbfc4
vfs: extract 'vfs' class and related code to a new 'vfs' module (API)
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
31045
diff
changeset
|
1 # vfs.py - Mercurial 'vfs' classes |
13962
8b252e826c68
add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff
changeset
|
2 # |
46819
d4ba4d51f85f
contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents:
45957
diff
changeset
|
3 # Copyright Olivia Mackall <olivia@selenic.com> |
13962
8b252e826c68
add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff
changeset
|
4 # |
8b252e826c68
add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
8b252e826c68
add: introduce a warning message for non-portable filenames (issue2756) (BC)
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
27482
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
7 |
51901
f4733654f144
typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents:
51739
diff
changeset
|
8 from __future__ import annotations |
f4733654f144
typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents:
51739
diff
changeset
|
9 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
10 import abc |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
11 import contextlib |
27482
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
12 import os |
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
13 import shutil |
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
14 import stat |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
15 import threading |
27482
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
16 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
17 from typing import ( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
18 Optional, |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
19 ) |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
20 |
27482
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
21 from .i18n import _ |
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
22 from . import ( |
34040
d5b2beca16c0
python3: wrap all uses of <exception>.strerror with strtolocal
Augie Fackler <raf@durin42.com>
parents:
33647
diff
changeset
|
23 encoding, |
27482
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
24 error, |
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
25 pathutil, |
30318
af7c60988f6e
py3: make scmutil.rcpath() return bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30109
diff
changeset
|
26 pycompat, |
27482
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
27 util, |
dde3da2246f1
scmutil: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27093
diff
changeset
|
28 ) |
18690
4c6f7f0dadab
scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
18678
diff
changeset
|
29 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
30 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
31 def _avoidambig(path: bytes, oldstat): |
33280
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
32 """Avoid file stat ambiguity forcibly |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
33 |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
34 This function causes copying ``path`` file, if it is owned by |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
35 another (see issue5418 and issue5584 for detail). |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
36 """ |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
37 |
33280
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
38 def checkandavoid(): |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
39 newstat = util.filestat.frompath(path) |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
40 # return whether file stat ambiguity is (already) avoided |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
41 return not newstat.isambig(oldstat) or newstat.avoidambig(path, oldstat) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
42 |
33280
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
43 if not checkandavoid(): |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
44 # simply copy to change owner of path to get privilege to |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
45 # advance mtime (see issue5418) |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
46 util.rename(util.mktempcopy(path), path) |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
47 checkandavoid() |
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
48 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
49 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
50 class abstractvfs(abc.ABC): |
14089
d3f7e110c3c0
opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14068
diff
changeset
|
51 """Abstract base class; cannot be instantiated""" |
d3f7e110c3c0
opener: introduce an abstact superclass of it
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14068
diff
changeset
|
52 |
47804
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
53 # default directory separator for vfs |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
54 # |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
55 # Other vfs code always use `/` and this works fine because python file API |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
56 # abstract the use of `/` and make it work transparently. For consistency |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
57 # vfs will always use `/` when joining. This avoid some confusion in |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
58 # encoded vfs (see issue6546) |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
59 _dir_sep = b'/' |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
60 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
61 # TODO: type return, which is util.posixfile wrapped by a proxy |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
62 @abc.abstractmethod |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
63 def __call__(self, path: bytes, mode: bytes = b'rb', **kwargs): |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
64 ... |
43515
b16912f8c07c
vfs: add a NotImplementedError implementation of __call__
Augie Fackler <augie@google.com>
parents:
43514
diff
changeset
|
65 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
66 @abc.abstractmethod |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
67 def _auditpath(self, path: bytes, mode: bytes): |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
68 ... |
41094
d9b6b9ed96d8
vfs: add a `_auditpath` to abstract vfs
Boris Feld <boris.feld@octobus.net>
parents:
41093
diff
changeset
|
69 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
70 @abc.abstractmethod |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
71 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes: |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
72 ... |
43516
8b0fa4de0064
vfs: add NotImplementedError version of join
Augie Fackler <augie@google.com>
parents:
43515
diff
changeset
|
73 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
74 def tryread(self, path: bytes) -> bytes: |
16479
fc04698fa778
opener: coding style, use triple quotes for doc string
Thomas Arendsen Hein <thomas@intevation.de>
parents:
16455
diff
changeset
|
75 '''gracefully return an empty string for missing files''' |
16455
154219f3a6a4
opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents:
16436
diff
changeset
|
76 try: |
154219f3a6a4
opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents:
16436
diff
changeset
|
77 return self.read(path) |
49314
2e726c934fcd
py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents:
49037
diff
changeset
|
78 except FileNotFoundError: |
2e726c934fcd
py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents:
49037
diff
changeset
|
79 pass |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
80 return b"" |
16455
154219f3a6a4
opener: introduce tryread helper
Matt Mackall <mpm@selenic.com>
parents:
16436
diff
changeset
|
81 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
82 def tryreadlines(self, path: bytes, mode: bytes = b'rb'): |
23368
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
83 '''gracefully return an empty array for missing files''' |
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
84 try: |
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
85 return self.readlines(path, mode=mode) |
49314
2e726c934fcd
py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents:
49037
diff
changeset
|
86 except FileNotFoundError: |
2e726c934fcd
py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents:
49037
diff
changeset
|
87 pass |
23368
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
88 return [] |
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
89 |
29730
2dd8c225e94c
vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
29727
diff
changeset
|
90 @util.propertycache |
2dd8c225e94c
vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
29727
diff
changeset
|
91 def open(self): |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
92 """Open ``path`` file, which is relative to vfs root. |
23370
46265d0f0c7b
vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23368
diff
changeset
|
93 |
46265d0f0c7b
vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23368
diff
changeset
|
94 Newly created directories are marked as "not to be indexed by |
46265d0f0c7b
vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23368
diff
changeset
|
95 the content indexing service", if ``notindexed`` is specified |
46265d0f0c7b
vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23368
diff
changeset
|
96 for "write" mode access. |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
97 """ |
29730
2dd8c225e94c
vfs: use propertycache for open
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
29727
diff
changeset
|
98 return self.__call__ |
19897
896a4568def7
vfs: add "open()" for newly added code paths which open files via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19896
diff
changeset
|
99 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
100 def read(self, path: bytes) -> bytes: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
101 with self(path, b'rb') as fp: |
14097
ca3376f044f8
opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14090
diff
changeset
|
102 return fp.read() |
ca3376f044f8
opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14090
diff
changeset
|
103 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
104 def readlines(self, path: bytes, mode: bytes = b'rb'): |
27706
22e362da27cf
scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27651
diff
changeset
|
105 with self(path, mode=mode) as fp: |
23368
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
106 return fp.readlines() |
bf8c3172255c
vfs: add "readlines" and "tryreadlines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23259
diff
changeset
|
107 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
108 def write( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
109 self, path: bytes, data: bytes, backgroundclose=False, **kwargs |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
110 ) -> int: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
111 with self(path, b'wb', backgroundclose=backgroundclose, **kwargs) as fp: |
14167
0e4753807c93
util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14138
diff
changeset
|
112 return fp.write(data) |
0e4753807c93
util & scmutil: adapt read/write helpers as request by mpm
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14138
diff
changeset
|
113 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
114 def writelines( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
115 self, path: bytes, data: bytes, mode: bytes = b'wb', notindexed=False |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
116 ) -> None: |
27706
22e362da27cf
scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27651
diff
changeset
|
117 with self(path, mode=mode, notindexed=notindexed) as fp: |
23371
1df6519eb3ab
vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23370
diff
changeset
|
118 return fp.writelines(data) |
1df6519eb3ab
vfs: add "writelines"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23370
diff
changeset
|
119 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
120 def append(self, path: bytes, data: bytes) -> int: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
121 with self(path, b'ab') as fp: |
14097
ca3376f044f8
opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14090
diff
changeset
|
122 return fp.write(data) |
ca3376f044f8
opener: add read & write utility methods
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14090
diff
changeset
|
123 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
124 def basename(self, path: bytes) -> bytes: |
25770
39de2e9cc6f4
vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25739
diff
changeset
|
125 """return base element of a path (as os.path.basename would do) |
39de2e9cc6f4
vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25739
diff
changeset
|
126 |
39de2e9cc6f4
vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25739
diff
changeset
|
127 This exists to allow handling of strange encoding if needed.""" |
39de2e9cc6f4
vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25739
diff
changeset
|
128 return os.path.basename(path) |
39de2e9cc6f4
vfs: add basename
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25739
diff
changeset
|
129 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
130 def chmod(self, path: bytes, mode: int) -> None: |
20086
f3df2612f3c3
vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20085
diff
changeset
|
131 return os.chmod(self.join(path), mode) |
f3df2612f3c3
vfs: add "chmod()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20085
diff
changeset
|
132 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
133 def dirname(self, path: bytes) -> bytes: |
25772
5471965af5cb
vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25770
diff
changeset
|
134 """return dirname element of a path (as os.path.dirname would do) |
5471965af5cb
vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25770
diff
changeset
|
135 |
5471965af5cb
vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25770
diff
changeset
|
136 This exists to allow handling of strange encoding if needed.""" |
5471965af5cb
vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25770
diff
changeset
|
137 return os.path.dirname(path) |
5471965af5cb
vfs: add dirname
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25770
diff
changeset
|
138 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
139 def exists(self, path: Optional[bytes] = None) -> bool: |
17161
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
140 return os.path.exists(self.join(path)) |
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
141 |
19899
8c3dcbbfb5de
changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19897
diff
changeset
|
142 def fstat(self, fp): |
8c3dcbbfb5de
changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19897
diff
changeset
|
143 return util.fstat(fp) |
8c3dcbbfb5de
changelog: use "vfs.fstat()" instead of "util.fstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19897
diff
changeset
|
144 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
145 def isdir(self, path: Optional[bytes] = None) -> bool: |
17161
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
146 return os.path.isdir(self.join(path)) |
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
147 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
148 def isfile(self, path: Optional[bytes] = None) -> bool: |
20085
589d6bb5b18d
vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20045
diff
changeset
|
149 return os.path.isfile(self.join(path)) |
589d6bb5b18d
vfs: add "isfile()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20045
diff
changeset
|
150 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
151 def islink(self, path: Optional[bytes] = None) -> bool: |
18949
138978f20180
localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18948
diff
changeset
|
152 return os.path.islink(self.join(path)) |
138978f20180
localrepo: use "vfs.islink()" instead of "os.path.islink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18948
diff
changeset
|
153 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
154 def isfileorlink(self, path: Optional[bytes] = None) -> bool: |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
155 """return whether path is a regular file or a symlink |
27571
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
156 |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
157 Unlike isfile, this doesn't follow symlinks.""" |
27571
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
158 try: |
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
159 st = self.lstat(path) |
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
160 except OSError: |
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
161 return False |
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
162 mode = st.st_mode |
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
163 return stat.S_ISREG(mode) or stat.S_ISLNK(mode) |
6a6e78f84cc6
merge: while checking for unknown files don't follow symlinks (issue5027)
Siddharth Agarwal <sid0@fb.com>
parents:
26836
diff
changeset
|
164 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
165 def _join(self, *paths: bytes) -> bytes: |
47804
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
166 root_idx = 0 |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
167 for idx, p in enumerate(paths): |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
168 if os.path.isabs(p) or p.startswith(self._dir_sep): |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
169 root_idx = idx |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
170 if root_idx != 0: |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
171 paths = paths[root_idx:] |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
172 paths = [p for p in paths if p] |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
173 return self._dir_sep.join(paths) |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
174 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
175 def reljoin(self, *paths: bytes) -> bytes: |
23581
aed981c7bebf
vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23539
diff
changeset
|
176 """join various elements of a path together (as os.path.join would do) |
aed981c7bebf
vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23539
diff
changeset
|
177 |
aed981c7bebf
vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23539
diff
changeset
|
178 The vfs base is not injected so that path stay relative. This exists |
aed981c7bebf
vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23539
diff
changeset
|
179 to allow handling of strange encoding if needed.""" |
47804
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
180 return self._join(*paths) |
23581
aed981c7bebf
vfs: add a 'reljoin' function for joining relative paths
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23539
diff
changeset
|
181 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
182 def split(self, path: bytes): |
23582
7559dc8c4238
vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23581
diff
changeset
|
183 """split top-most element of a path (as os.path.split would do) |
7559dc8c4238
vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23581
diff
changeset
|
184 |
7559dc8c4238
vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23581
diff
changeset
|
185 This exists to allow handling of strange encoding if needed.""" |
7559dc8c4238
vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23581
diff
changeset
|
186 return os.path.split(path) |
7559dc8c4238
vfs: add a 'split' method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23581
diff
changeset
|
187 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
188 def lexists(self, path: Optional[bytes] = None) -> bool: |
21563
764b691b8bda
vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents:
21111
diff
changeset
|
189 return os.path.lexists(self.join(path)) |
764b691b8bda
vfs: add lexists() in current api
Chinmay Joshi <c@chinmayjoshi.com>
parents:
21111
diff
changeset
|
190 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
191 def lstat(self, path: Optional[bytes] = None): |
19900
7c21e3398931
context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19899
diff
changeset
|
192 return os.lstat(self.join(path)) |
7c21e3398931
context: use "vfs.lstat()" instead of "os.lstat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19899
diff
changeset
|
193 |
51649
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
194 def is_mmap_safe(self, path: Optional[bytes] = None) -> bool: |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
195 """return True if it is safe to read a file content as mmap |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
196 |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
197 This focus on the file system aspect of such safety, the application |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
198 logic around that file is not taken into account, so caller need to |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
199 make sure the file won't be truncated in a way that will create SIGBUS |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
200 on access. |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
201 |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
202 |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
203 The initial motivation for this logic is that if mmap is used on NFS |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
204 and somebody deletes the mapped file (e.g. by renaming on top of it), |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
205 then you get SIGBUS, which can be pretty disruptive: we get core dump |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
206 reports, and the process terminates without writing to the blackbox. |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
207 |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
208 Instead in this situation we prefer to read the file normally. |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
209 The risk of ESTALE in the middle of the read remains, but it's |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
210 smaller because we read sooner and the error should be reported |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
211 just as any other error. |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
212 |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
213 Note that python standard library does not offer the necessary function |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
214 to detect the file stem bits. So this detection rely on compiled bits |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
215 and is not available in pure python. |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
216 """ |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
217 # XXX Since we already assume a vfs to address a consistent file system |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
218 # in other location, we could determine the fstype once for the root |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
219 # and cache that value. |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
220 fstype = util.getfstype(self.join(path)) |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
221 return fstype is not None and fstype != b'nfs' |
ba205f944cb4
mmap: add a `is_mmap_safe` method to vfs
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51618
diff
changeset
|
222 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
223 def listdir(self, path: Optional[bytes] = None): |
21799
dfacdd6a111e
vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents:
21716
diff
changeset
|
224 return os.listdir(self.join(path)) |
dfacdd6a111e
vfs: add listdir for os.listdir in vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents:
21716
diff
changeset
|
225 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
226 def makedir(self, path: Optional[bytes] = None, notindexed=True): |
17161
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
227 return util.makedir(self.join(path), notindexed) |
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
228 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
229 def makedirs( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
230 self, path: Optional[bytes] = None, mode: Optional[int] = None |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
231 ): |
17161
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
232 return util.makedirs(self.join(path), mode) |
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
233 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
234 def makelock(self, info, path: bytes): |
20090
88d8e568add1
vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20086
diff
changeset
|
235 return util.makelock(info, self.join(path)) |
88d8e568add1
vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20086
diff
changeset
|
236 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
237 def mkdir(self, path: Optional[bytes] = None): |
17723
ab23768746fd
scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17681
diff
changeset
|
238 return os.mkdir(self.join(path)) |
ab23768746fd
scmutil: reorder newly added functions for vfs support in dictionary order
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17681
diff
changeset
|
239 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
240 def mkstemp( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
241 self, |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
242 suffix: bytes = b'', |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
243 prefix: bytes = b'tmp', |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
244 dir: Optional[bytes] = None, |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
245 ): |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
246 fd, name = pycompat.mkstemp( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
247 suffix=suffix, prefix=prefix, dir=self.join(dir) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
248 ) |
20980
6fb4c94ae300
vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20862
diff
changeset
|
249 dname, fname = util.split(name) |
6fb4c94ae300
vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20862
diff
changeset
|
250 if dir: |
6fb4c94ae300
vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20862
diff
changeset
|
251 return fd, os.path.join(dir, fname) |
6fb4c94ae300
vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20862
diff
changeset
|
252 else: |
6fb4c94ae300
vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20862
diff
changeset
|
253 return fd, fname |
6fb4c94ae300
vfs: add "mkstemp()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20862
diff
changeset
|
254 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
255 def readdir(self, path: Optional[bytes] = None, stat=None, skip=None): |
32248
d74b0cff94a9
osutil: proxy through util (and platform) modules (API)
Yuya Nishihara <yuya@tcha.org>
parents:
31649
diff
changeset
|
256 return util.listdir(self.join(path), stat, skip) |
17747
aad3bce98f76
store: invoke "osutil.listdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17726
diff
changeset
|
257 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
258 def readlock(self, path: bytes) -> bytes: |
20090
88d8e568add1
vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20086
diff
changeset
|
259 return util.readlock(self.join(path)) |
88d8e568add1
vfs: add "makelock()" and "readlock()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
20086
diff
changeset
|
260 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
261 def rename(self, src: bytes, dst: bytes, checkambig=False): |
29367
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
262 """Rename from src to dst |
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
263 |
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
264 checkambig argument is used with util.filestat, and is useful |
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
265 only if destination file is guarded by any lock |
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
266 (e.g. repo.lock or repo.wlock). |
33282
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
267 |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
268 To avoid file stat ambiguity forcibly, checkambig=True involves |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
269 copying ``src`` file, if it is owned by another. Therefore, use |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
270 checkambig=True only in limited cases (see also issue5418 and |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
271 issue5584 for detail). |
29367
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
272 """ |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
273 self._auditpath(dst, b'w') |
32768
ed66ec39933f
vfs: create copy at renaming to avoid file stat ambiguity if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
32767
diff
changeset
|
274 srcpath = self.join(src) |
29203
731ced087a4b
vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29202
diff
changeset
|
275 dstpath = self.join(dst) |
32792
7ad95626f6a7
filestat: move __init__ to frompath constructor
Siddharth Agarwal <sid0@fb.com>
parents:
32768
diff
changeset
|
276 oldstat = checkambig and util.filestat.frompath(dstpath) |
29203
731ced087a4b
vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29202
diff
changeset
|
277 if oldstat and oldstat.stat: |
33281
6af0f023d014
vfs: replace avoiding ambiguity in abstractvfs.rename with _avoidambig
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33280
diff
changeset
|
278 ret = util.rename(srcpath, dstpath) |
6af0f023d014
vfs: replace avoiding ambiguity in abstractvfs.rename with _avoidambig
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33280
diff
changeset
|
279 _avoidambig(dstpath, oldstat) |
29203
731ced087a4b
vfs: make rename avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29202
diff
changeset
|
280 return ret |
32768
ed66ec39933f
vfs: create copy at renaming to avoid file stat ambiguity if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
32767
diff
changeset
|
281 return util.rename(srcpath, dstpath) |
18948
2f05fa162316
localrepo: use "vfs.rename()" instead of "util.rename()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18945
diff
changeset
|
282 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
283 def readlink(self, path: bytes) -> bytes: |
39909
5fe0b880200e
py3: convert os.readlink() path to native strings on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
39481
diff
changeset
|
284 return util.readlink(self.join(path)) |
18950
647e3b0c8751
localrepo: use "vfs.readlink()" instead of "os.readlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18949
diff
changeset
|
285 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
286 def removedirs(self, path: Optional[bytes] = None): |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
287 """Remove a leaf directory and all empty intermediate ones""" |
24693
0d28b0df77ea
vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24689
diff
changeset
|
288 return util.removedirs(self.join(path)) |
0d28b0df77ea
vfs: add removedirs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24689
diff
changeset
|
289 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
290 def rmdir(self, path: Optional[bytes] = None): |
39481
3dd34b401bc2
merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38497
diff
changeset
|
291 """Remove an empty directory.""" |
3dd34b401bc2
merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38497
diff
changeset
|
292 return os.rmdir(self.join(path)) |
3dd34b401bc2
merge: use vfs methods for I/O
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38497
diff
changeset
|
293 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
294 def rmtree( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
295 self, path: Optional[bytes] = None, ignore_errors=False, forcibly=False |
51739
be6d8ea6d3d2
typing: add a trivial type hint to `mercurial/vfs.py`
Matt Harbison <matt_harbison@yahoo.com>
parents:
51649
diff
changeset
|
296 ) -> None: |
24689
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
297 """Remove a directory tree recursively |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
298 |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
299 If ``forcibly``, this tries to remove READ-ONLY files, too. |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
300 """ |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
301 if forcibly: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
302 |
50750
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
303 def onexc(function, path, excinfo): |
24689
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
304 if function is not os.remove: |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
305 raise |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
306 # read-only files cannot be unlinked under Windows |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
307 s = os.stat(path) |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
308 if (s.st_mode & stat.S_IWRITE) != 0: |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
309 raise |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
310 os.chmod(path, stat.S_IMODE(s.st_mode) | stat.S_IWRITE) |
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
311 os.remove(path) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
312 |
24689
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
313 else: |
50750
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
314 onexc = None |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
315 try: |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
316 # pytype: disable=wrong-keyword-args |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
317 return shutil.rmtree( |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
318 self.join(path), ignore_errors=ignore_errors, onexc=onexc |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
319 ) |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
320 # pytype: enable=wrong-keyword-args |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
321 except TypeError: # onexc was introduced in Python 3.12 |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
322 return shutil.rmtree( |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
323 self.join(path), ignore_errors=ignore_errors, onerror=onexc |
f173c2c23289
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12
Mads Kiilerich <mads@kiilerich.com>
parents:
50365
diff
changeset
|
324 ) |
24689
ca3a90096c95
vfs: add rmtree
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24635
diff
changeset
|
325 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
326 def setflags(self, path: bytes, l: bool, x: bool): |
18951
d13916a00b7e
localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18950
diff
changeset
|
327 return util.setflags(self.join(path), l, x) |
d13916a00b7e
localrepo: use "vfs.setflags()" instead of "util.setflags()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18950
diff
changeset
|
328 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
329 def stat(self, path: Optional[bytes] = None): |
17726
7cb7e17c23b2
store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17725
diff
changeset
|
330 return os.stat(self.join(path)) |
7cb7e17c23b2
store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17725
diff
changeset
|
331 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
332 def unlink(self, path: Optional[bytes] = None): |
19895
37c0d93fb166
bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19655
diff
changeset
|
333 return util.unlink(self.join(path)) |
37c0d93fb166
bookmarks: use "vfs.unlink()" instead of "util.unlink()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19655
diff
changeset
|
334 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
335 def tryunlink(self, path: Optional[bytes] = None): |
31548 | 336 """Attempt to remove a file, ignoring missing file errors.""" |
51428
187c5769a629
vfs: have tryunlink tell what it did
Georges Racinet <georges.racinet@octobus.net>
parents:
50952
diff
changeset
|
337 return util.tryunlink(self.join(path)) |
31548 | 338 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
339 def unlinkpath( |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
340 self, path: Optional[bytes] = None, ignoremissing=False, rmdir=True |
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
341 ): |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
342 return util.unlinkpath( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
343 self.join(path), ignoremissing=ignoremissing, rmdir=rmdir |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
344 ) |
21716
90f9be5adade
vfs: add unlinkpath to vfs
Chinmay Joshi <c@chinmayjoshi.com>
parents:
21563
diff
changeset
|
345 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
346 def utime(self, path: Optional[bytes] = None, t=None): |
19896
af03279c766a
bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19895
diff
changeset
|
347 return os.utime(self.join(path), t) |
af03279c766a
bookmarks: use "vfs.utime()" instead of "os.utime()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19895
diff
changeset
|
348 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
349 def walk(self, path: Optional[bytes] = None, onerror=None): |
24725 | 350 """Yield (dirpath, dirs, files) tuple for each directories under path |
351 | |
352 ``dirpath`` is relative one from the root of this vfs. This | |
353 uses ``os.sep`` as path separator, even you specify POSIX | |
354 style ``path``. | |
355 | |
356 "The root of this vfs" is represented as empty ``dirpath``. | |
357 """ | |
358 root = os.path.normpath(self.join(None)) | |
359 # when dirpath == root, dirpath[prefixlen:] becomes empty | |
360 # because len(dirpath) < prefixlen. | |
361 prefixlen = len(pathutil.normasprefix(root)) | |
362 for dirpath, dirs, files in os.walk(self.join(path), onerror=onerror): | |
363 yield (dirpath[prefixlen:], dirs, files) | |
364 | |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
365 @contextlib.contextmanager |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
366 def backgroundclosing(self, ui, expectedcount=-1): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
367 """Allow files to be closed asynchronously. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
368 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
369 When this context manager is active, ``backgroundclose`` can be passed |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
370 to ``__call__``/``open`` to result in the file possibly being closed |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
371 asynchronously, on a background thread. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
372 """ |
35432
60f2a215faa7
workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents:
34645
diff
changeset
|
373 # Sharing backgroundfilecloser between threads is complex and using |
60f2a215faa7
workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents:
34645
diff
changeset
|
374 # multiple instances puts us at risk of running out of file descriptors |
60f2a215faa7
workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents:
34645
diff
changeset
|
375 # only allow to use backgroundfilecloser when in main thread. |
51920
1edac12af730
vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents:
51901
diff
changeset
|
376 if threading.current_thread() is not threading.main_thread(): |
35432
60f2a215faa7
workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents:
34645
diff
changeset
|
377 yield |
60f2a215faa7
workers: don't use backgroundfilecloser in threads
Wojciech Lis <wlis@fb.com>
parents:
34645
diff
changeset
|
378 return |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
379 vfs = getattr(self, 'vfs', self) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
380 if getattr(vfs, '_backgroundfilecloser', None): |
29389
98e8313dcd9e
i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents:
29373
diff
changeset
|
381 raise error.Abort( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
382 _(b'can only have 1 active background file closer') |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
383 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
384 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
385 with backgroundfilecloser(ui, expectedcount=expectedcount) as bfc: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
386 try: |
43530
fc19f8ab8199
vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents:
43516
diff
changeset
|
387 vfs._backgroundfilecloser = ( |
fc19f8ab8199
vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents:
43516
diff
changeset
|
388 bfc # pytype: disable=attribute-error |
fc19f8ab8199
vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents:
43516
diff
changeset
|
389 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
390 yield bfc |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
391 finally: |
43530
fc19f8ab8199
vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents:
43516
diff
changeset
|
392 vfs._backgroundfilecloser = ( |
fc19f8ab8199
vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents:
43516
diff
changeset
|
393 None # pytype: disable=attribute-error |
fc19f8ab8199
vfs: suppress some pytype errors around us using a private attribute
Augie Fackler <augie@google.com>
parents:
43516
diff
changeset
|
394 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
395 |
47449
9ab54aa56982
vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47192
diff
changeset
|
396 def register_file(self, path): |
9ab54aa56982
vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47192
diff
changeset
|
397 """generic hook point to lets fncache steer its stew""" |
9ab54aa56982
vfs: add a `register_file` method on the vfs class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47192
diff
changeset
|
398 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
399 |
17649
f65c6a5f256c
scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17561
diff
changeset
|
400 class vfs(abstractvfs): |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
401 """Operate files relative to a base directory |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
402 |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
403 This class is used to hide the details of COW semantics and |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
404 remote file access from higher level code. |
33647
377e8ddaebef
pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents:
33435
diff
changeset
|
405 |
377e8ddaebef
pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents:
33435
diff
changeset
|
406 'cacheaudited' should be enabled only if (a) vfs object is short-lived, or |
377e8ddaebef
pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents:
33435
diff
changeset
|
407 (b) the base directory is managed by hg and considered sort-of append-only. |
377e8ddaebef
pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents:
33435
diff
changeset
|
408 See pathutil.pathauditor() for details. |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
409 """ |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
410 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
411 def __init__( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
412 self, |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
413 base: bytes, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
414 audit=True, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
415 cacheaudited=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
416 expandpath=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
417 realpath=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
418 ): |
18945
e75b72fffdfe
vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18900
diff
changeset
|
419 if expandpath: |
e75b72fffdfe
vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18900
diff
changeset
|
420 base = util.expandpath(base) |
e75b72fffdfe
vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18900
diff
changeset
|
421 if realpath: |
e75b72fffdfe
vfs: split "expand" into "realpath"/"expandpath" to apply each separately
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
18900
diff
changeset
|
422 base = os.path.realpath(base) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
423 self.base = base |
33259
6fb5c5096887
vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33257
diff
changeset
|
424 self._audit = audit |
6fb5c5096887
vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33257
diff
changeset
|
425 if audit: |
33647
377e8ddaebef
pathauditor: disable cache of audited paths by default (issue5628)
Yuya Nishihara <yuya@tcha.org>
parents:
33435
diff
changeset
|
426 self.audit = pathutil.pathauditor(self.base, cached=cacheaudited) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
427 else: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
428 self.audit = lambda path, mode=None: True |
33259
6fb5c5096887
vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33257
diff
changeset
|
429 self.createmode = None |
6fb5c5096887
vfs: drop the 'mustaudit' API
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33257
diff
changeset
|
430 self._trustnlink = None |
43027
3518da504303
vfs: give all vfs an options attribute by default
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41098
diff
changeset
|
431 self.options = {} |
17554
5450c8ad9d98
scmutil: turn opener._audit into a property, mustaudit
Bryan O'Sullivan <bryano@fb.com>
parents:
17248
diff
changeset
|
432 |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
433 @util.propertycache |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
434 def _cansymlink(self) -> bool: |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
435 return util.checklink(self.base) |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
436 |
18192
f9a89bdd64a6
scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents:
17850
diff
changeset
|
437 @util.propertycache |
f9a89bdd64a6
scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents:
17850
diff
changeset
|
438 def _chmod(self): |
f9a89bdd64a6
scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents:
17850
diff
changeset
|
439 return util.checkexec(self.base) |
f9a89bdd64a6
scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents:
17850
diff
changeset
|
440 |
17763
13070de77c86
vfs: backout fchmod change from 76b73ce0ffac
Matt Mackall <mpm@selenic.com>
parents:
17752
diff
changeset
|
441 def _fixfilemode(self, name): |
18192
f9a89bdd64a6
scmutil: don't try to match modes on filesystems without modes (issue3740)
Matt Mackall <mpm@selenic.com>
parents:
17850
diff
changeset
|
442 if self.createmode is None or not self._chmod: |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
443 return |
25658
e93036747902
global: mass rewrite to use modern octal syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25629
diff
changeset
|
444 os.chmod(name, self.createmode & 0o666) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
445 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
446 def _auditpath(self, path, mode) -> None: |
40788
34f15db81cf0
vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents:
39909
diff
changeset
|
447 if self._audit: |
41093
c8006a25b845
vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents:
40999
diff
changeset
|
448 if os.path.isabs(path) and path.startswith(self.base): |
c8006a25b845
vfs: makes all audited path relative
Boris Feld <boris.feld@octobus.net>
parents:
40999
diff
changeset
|
449 path = os.path.relpath(path, self.base) |
40788
34f15db81cf0
vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents:
39909
diff
changeset
|
450 r = util.checkosfilename(path) |
34f15db81cf0
vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents:
39909
diff
changeset
|
451 if r: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
452 raise error.Abort(b"%s: %r" % (r, path)) |
40788
34f15db81cf0
vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents:
39909
diff
changeset
|
453 self.audit(path, mode=mode) |
34f15db81cf0
vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents:
39909
diff
changeset
|
454 |
49982
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
455 def isfileorlink_checkdir( |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
456 self, dircache, path: Optional[bytes] = None |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
457 ) -> bool: |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
458 """return True if the path is a regular file or a symlink and |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
459 the directories along the path are "normal", that is |
49985
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
460 not symlinks or nested hg repositories. |
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
461 |
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
462 Ignores the `_audit` setting, and checks the directories regardless. |
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
463 `dircache` is used to cache the directory checks. |
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
464 """ |
49982
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
465 try: |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
466 for prefix in pathutil.finddirs_rev_noroot(util.localpath(path)): |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
467 if prefix in dircache: |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
468 res = dircache[prefix] |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
469 else: |
49985
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
470 res = pathutil.pathauditor._checkfs_exists( |
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
471 self.base, prefix, path |
bc83ebe07bf0
pathauditor: make _checkfs_exists a static method
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49982
diff
changeset
|
472 ) |
49982
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
473 dircache[prefix] = res |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
474 if not res: |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
475 return False |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
476 except (OSError, error.Abort): |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
477 return False |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
478 return self.isfileorlink(path) |
b7cf91ef03ba
merge: skip syntactic path checks in [_checkunknownfile]
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents:
49723
diff
changeset
|
479 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
480 def __call__( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
481 self, |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
482 path: bytes, |
49723
2506c3ac73f4
vfs: make the default opener mode binary
Matt Harbison <matt_harbison@yahoo.com>
parents:
49722
diff
changeset
|
483 mode: bytes = b"rb", |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
484 atomictemp=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
485 notindexed=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
486 backgroundclose=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
487 checkambig=False, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
488 auditpath=True, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
489 makeparentdirs=True, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
490 ): |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
491 """Open ``path`` file, which is relative to vfs root. |
23370
46265d0f0c7b
vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
23368
diff
changeset
|
492 |
40825
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
493 By default, parent directories are created as needed. Newly created |
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
494 directories are marked as "not to be indexed by the content indexing |
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
495 service", if ``notindexed`` is specified for "write" mode access. |
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
496 Set ``makeparentdirs=False`` to not create directories implicitly. |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
497 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
498 If ``backgroundclose`` is passed, the file may be closed asynchronously. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
499 It can only be used if the ``self.backgroundclosing()`` context manager |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
500 is active. This should only be specified if the following criteria hold: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
501 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
502 1. There is a potential for writing thousands of files. Unless you |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
503 are writing thousands of files, the performance benefits of |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
504 asynchronously closing files is not realized. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
505 2. Files are opened exactly once for the ``backgroundclosing`` |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
506 active duration and are therefore free of race conditions between |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
507 closing a file on a background thread and reopening it. (If the |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
508 file were opened multiple times, there could be unflushed data |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
509 because the original file handle hasn't been flushed/closed yet.) |
29202
76f1ea360c7e
vfs: make atomictempfile avoid ambiguity of file stat if needed
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29119
diff
changeset
|
510 |
44575
77d48738b8e0
vfs: fix typo in comment (remove extra "l")
Kyle Lippincott <spectral@google.com>
parents:
43554
diff
changeset
|
511 ``checkambig`` argument is passed to atomictempfile (valid |
29367
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
512 only for writing), and is useful only if target file is |
4e6e280e238f
doc: describe detail about checkambig optional argument
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29341
diff
changeset
|
513 guarded by any lock (e.g. repo.lock or repo.wlock). |
33282
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
514 |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
515 To avoid file stat ambiguity forcibly, checkambig=True involves |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
516 copying ``path`` file opened in "append" mode (e.g. for |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
517 truncation), if it is owned by another. Therefore, use |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
518 combination of append mode and checkambig=True only in limited |
d1db7af81548
vfs: add explanation about cost of checkambig=True in corner case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33281
diff
changeset
|
519 cases (see also issue5418 and issue5584 for detail). |
45957
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
44575
diff
changeset
|
520 """ |
33257
15e9cbe6ae49
vfs: simplify path audit disabling in stream clone
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33234
diff
changeset
|
521 if auditpath: |
40788
34f15db81cf0
vfs: extract the audit path logic into a submethod
Boris Feld <boris.feld@octobus.net>
parents:
39909
diff
changeset
|
522 self._auditpath(path, mode) |
16199
8181bd808dc5
scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents:
16198
diff
changeset
|
523 f = self.join(path) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
524 |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
525 if b"b" not in mode: |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
526 mode += b"b" # for that other OS |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
527 |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
528 nlink = -1 |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
529 if mode not in (b'r', b'rb'): |
17937
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
530 dirname, basename = util.split(f) |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
531 # If basename is empty, then the path is malformed because it points |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
532 # to a directory. Let the posixfile() call below raise IOError. |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
533 if basename: |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
534 if atomictemp: |
40825
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
535 if makeparentdirs: |
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
536 util.makedirs(dirname, self.createmode, notindexed) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
537 return util.atomictempfile( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
538 f, mode, self.createmode, checkambig=checkambig |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
539 ) |
17937
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
540 try: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
541 if b'w' in mode: |
17937
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
542 util.unlink(f) |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
543 nlink = 0 |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
544 else: |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
545 # nlinks() may behave differently for files on Windows |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
546 # shares if the file is open. |
27706
22e362da27cf
scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27651
diff
changeset
|
547 with util.posixfile(f): |
22e362da27cf
scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27651
diff
changeset
|
548 nlink = util.nlinks(f) |
22e362da27cf
scmutil: use context managers for file handles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27651
diff
changeset
|
549 if nlink < 1: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
550 nlink = 2 # force mktempcopy (issue1922) |
49314
2e726c934fcd
py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents:
49037
diff
changeset
|
551 except FileNotFoundError: |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
552 nlink = 0 |
40825
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
553 if makeparentdirs: |
03bca908d9fb
vfs: add option to not create parent directories implicitly
Yuya Nishihara <yuya@tcha.org>
parents:
40788
diff
changeset
|
554 util.makedirs(dirname, self.createmode, notindexed) |
17937
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
555 if nlink > 0: |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
556 if self._trustnlink is None: |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
557 self._trustnlink = nlink > 1 or util.checknlink(f) |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
558 if nlink > 1 or not self._trustnlink: |
3cb032d50447
vfs: optimize __call__ by not calling util.split for reads
Adrian Buehlmann <adrian@cadifra.com>
parents:
17850
diff
changeset
|
559 util.rename(util.mktempcopy(f), f) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
560 fp = util.posixfile(f, mode) |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
561 if nlink == 0: |
17763
13070de77c86
vfs: backout fchmod change from 76b73ce0ffac
Matt Mackall <mpm@selenic.com>
parents:
17752
diff
changeset
|
562 self._fixfilemode(f) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
563 |
30000
9766d88c2465
vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29999
diff
changeset
|
564 if checkambig: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
565 if mode in (b'r', b'rb'): |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
566 raise error.Abort( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
567 _( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
568 b'implementation error: mode %s is not' |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
569 b' valid for checkambig=True' |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
570 ) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
571 % mode |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
572 ) |
30000
9766d88c2465
vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29999
diff
changeset
|
573 fp = checkambigatclosing(fp) |
9766d88c2465
vfs: use checkambigatclosing in checkambig=True but atomictemp=False case
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29999
diff
changeset
|
574 |
51920
1edac12af730
vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents:
51901
diff
changeset
|
575 if ( |
1edac12af730
vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents:
51901
diff
changeset
|
576 backgroundclose |
1edac12af730
vfs: modernize the detection of the main thread
Matt Harbison <matt_harbison@yahoo.com>
parents:
51901
diff
changeset
|
577 and threading.current_thread() is threading.main_thread() |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
578 ): |
43532
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
579 if ( |
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
580 not self._backgroundfilecloser # pytype: disable=attribute-error |
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
581 ): |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
582 raise error.Abort( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
583 _( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
584 b'backgroundclose can only be used when a ' |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
585 b'backgroundclosing context manager is active' |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
586 ) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
587 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
588 |
43532
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
589 fp = delayclosedfile( |
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
590 fp, |
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
591 self._backgroundfilecloser, # pytype: disable=attribute-error |
da9ecbb10368
vfs: more attribute suppressions
Augie Fackler <augie@google.com>
parents:
43530
diff
changeset
|
592 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
593 |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
594 return fp |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
595 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
596 def symlink(self, src: bytes, dst: bytes) -> None: |
18327
4aecdb91443c
scmutil: simplify vfs.audit - drop wrapped vfs.auditor
Mads Kiilerich <mads@kiilerich.com>
parents:
18316
diff
changeset
|
597 self.audit(dst) |
16199
8181bd808dc5
scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents:
16198
diff
changeset
|
598 linkname = self.join(dst) |
31555 | 599 util.tryunlink(linkname) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
600 |
29017
07be86828e79
util: fix race in makedirs()
Adam Simpkins <simpkins@fb.com>
parents:
28819
diff
changeset
|
601 util.makedirs(os.path.dirname(linkname), self.createmode) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
602 |
14261
e3649bcca3f6
opener: rename _can_symlink to _cansymlink
Adrian Buehlmann <adrian@cadifra.com>
parents:
14236
diff
changeset
|
603 if self._cansymlink: |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
604 try: |
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
605 os.symlink(src, linkname) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25658
diff
changeset
|
606 except OSError as err: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
607 raise OSError( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
608 err.errno, |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
609 _(b'could not symlink to %r: %s') |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
610 % (src, encoding.strtolocal(err.strerror)), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
611 linkname, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
612 ) |
13970
d13913355390
move opener from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13962
diff
changeset
|
613 else: |
17768
9837cafc25b1
vfs: use self.write to write symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
17763
diff
changeset
|
614 self.write(dst, src) |
13971
bfeaa88b875d
move canonpath from util to scmutil
Adrian Buehlmann <adrian@cadifra.com>
parents:
13970
diff
changeset
|
615 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
616 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes: |
17161
be016e96117a
localrepo: use file API via vfs while ensuring repository directory
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17157
diff
changeset
|
617 if path: |
47804
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
618 parts = [self.base, path] |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
619 parts.extend(insidef) |
98c3fa6a3ac2
vfs: always use / as file separator (issue6546)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47449
diff
changeset
|
620 return self._join(*parts) |
17681
a41fd730f230
scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents:
17675
diff
changeset
|
621 else: |
a41fd730f230
scmutil: backout 83785bb56062 (issue3643)
Matt Mackall <mpm@selenic.com>
parents:
17675
diff
changeset
|
622 return self.base |
16199
8181bd808dc5
scmutil: add join method to opener to construct path relative to base
Idan Kamara <idankk86@gmail.com>
parents:
16198
diff
changeset
|
623 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
624 |
17649
f65c6a5f256c
scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17561
diff
changeset
|
625 opener = vfs |
f65c6a5f256c
scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17561
diff
changeset
|
626 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
627 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
628 class proxyvfs(abstractvfs, abc.ABC): |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
629 def __init__(self, vfs: "vfs"): |
17845
408ded42c5ec
scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents:
17821
diff
changeset
|
630 self.vfs = vfs |
408ded42c5ec
scmutil: abstract out mustaudit delegation
Bryan O'Sullivan <bryano@fb.com>
parents:
17821
diff
changeset
|
631 |
50365
d1d458fb96a5
vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49985
diff
changeset
|
632 @property |
d1d458fb96a5
vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49985
diff
changeset
|
633 def createmode(self): |
d1d458fb96a5
vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49985
diff
changeset
|
634 return self.vfs.createmode |
d1d458fb96a5
vfsproxy: inherit the `createmode` attribute too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
49985
diff
changeset
|
635 |
41097
adee334d94cd
vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents:
41096
diff
changeset
|
636 def _auditpath(self, path, mode): |
adee334d94cd
vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents:
41096
diff
changeset
|
637 return self.vfs._auditpath(path, mode) |
adee334d94cd
vfs: handle _auditpath in proxyvfs
Boris Feld <boris.feld@octobus.net>
parents:
41096
diff
changeset
|
638 |
27879
52a4ad62b006
cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents:
27851
diff
changeset
|
639 @property |
29727
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
640 def options(self): |
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
641 return self.vfs.options |
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
642 |
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
643 @options.setter |
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
644 def options(self, value): |
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
645 self.vfs.options = value |
69109052d9ac
auditvfs: forward options property from nested vfs
Augie Fackler <augie@google.com>
parents:
29417
diff
changeset
|
646 |
51618
b5500857e173
proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51428
diff
changeset
|
647 @property |
b5500857e173
proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51428
diff
changeset
|
648 def audit(self): |
b5500857e173
proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51428
diff
changeset
|
649 return self.vfs.audit |
b5500857e173
proxy-vfs: also proxy the `audit` attribute
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
51428
diff
changeset
|
650 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
651 |
41096
6498f0e03526
vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents:
41095
diff
changeset
|
652 class filtervfs(proxyvfs, abstractvfs): |
17649
f65c6a5f256c
scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17561
diff
changeset
|
653 '''Wrapper vfs for filtering filenames with a function.''' |
14090
e24b5e3c2f27
add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14089
diff
changeset
|
654 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
655 def __init__(self, vfs: "vfs", filter): |
33412
a42369e04aee
vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents:
33282
diff
changeset
|
656 proxyvfs.__init__(self, vfs) |
14090
e24b5e3c2f27
add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14089
diff
changeset
|
657 self._filter = filter |
e24b5e3c2f27
add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14089
diff
changeset
|
658 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
659 def __call__(self, path: bytes, *args, **kwargs): |
17846
f42cf30873dc
scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents:
17845
diff
changeset
|
660 return self.vfs(self._filter(path), *args, **kwargs) |
14090
e24b5e3c2f27
add filteropener abstraction for store openers
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
14089
diff
changeset
|
661 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
662 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes: |
17725
ffd589d4b785
vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17723
diff
changeset
|
663 if path: |
24628
a0b47885a1c5
vfs: make it possible to pass multiple path elements to join
Matt Harbison <matt_harbison@yahoo.com>
parents:
24582
diff
changeset
|
664 return self.vfs.join(self._filter(self.vfs.reljoin(path, *insidef))) |
17725
ffd589d4b785
vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17723
diff
changeset
|
665 else: |
17846
f42cf30873dc
scmutil: add mustaudit delegation to filtervfs (issue3673)
Bryan O'Sullivan <bryano@fb.com>
parents:
17845
diff
changeset
|
666 return self.vfs.join(path) |
17725
ffd589d4b785
vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17723
diff
changeset
|
667 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
668 |
17649
f65c6a5f256c
scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17561
diff
changeset
|
669 filteropener = filtervfs |
f65c6a5f256c
scmutil: rename classes from "opener" to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
17561
diff
changeset
|
670 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
671 |
41096
6498f0e03526
vfs: fix proxyvfs inheritance
Boris Feld <boris.feld@octobus.net>
parents:
41095
diff
changeset
|
672 class readonlyvfs(proxyvfs): |
18213
c38a62af000e
vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18206
diff
changeset
|
673 '''Wrapper vfs preventing any writing.''' |
c38a62af000e
vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18206
diff
changeset
|
674 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
675 def __init__(self, vfs: "vfs"): |
33412
a42369e04aee
vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents:
33282
diff
changeset
|
676 proxyvfs.__init__(self, vfs) |
18213
c38a62af000e
vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18206
diff
changeset
|
677 |
49723
2506c3ac73f4
vfs: make the default opener mode binary
Matt Harbison <matt_harbison@yahoo.com>
parents:
49722
diff
changeset
|
678 def __call__(self, path: bytes, mode: bytes = b'rb', *args, **kw): |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
679 if mode not in (b'r', b'rb'): |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
680 raise error.Abort(_(b'this vfs is read only')) |
18213
c38a62af000e
vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18206
diff
changeset
|
681 return self.vfs(path, mode, *args, **kw) |
c38a62af000e
vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18206
diff
changeset
|
682 |
49722
cc9a60050a07
typing: add basic type hints to vfs.py
Matt Harbison <matt_harbison@yahoo.com>
parents:
49314
diff
changeset
|
683 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes: |
26156
a112fffdb632
scmutil.readonlyvfs: implement join
Siddharth Agarwal <sid0@fb.com>
parents:
26098
diff
changeset
|
684 return self.vfs.join(path, *insidef) |
18213
c38a62af000e
vfs: add a read only vfs
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18206
diff
changeset
|
685 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
686 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
687 class closewrapbase(abc.ABC): |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
688 """Base class of wrapper, which hooks closing |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
689 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
690 Do not instantiate outside of the vfs layer. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
691 """ |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
692 |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
693 def __init__(self, fh): |
43554
9f70512ae2cf
cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents:
43532
diff
changeset
|
694 object.__setattr__(self, '_origfh', fh) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
695 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
696 def __getattr__(self, attr): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
697 return getattr(self._origfh, attr) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
698 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
699 def __setattr__(self, attr, value): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
700 return setattr(self._origfh, attr, value) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
701 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
702 def __delattr__(self, attr): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
703 return delattr(self._origfh, attr) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
704 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
705 def __enter__(self): |
40468
8d9f366b7f19
vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents:
39909
diff
changeset
|
706 self._origfh.__enter__() |
8d9f366b7f19
vfs: ensure closewrapbase fh doesn't escape by entering context manager
Matt Harbison <matt_harbison@yahoo.com>
parents:
39909
diff
changeset
|
707 return self |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
708 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
709 @abc.abstractmethod |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
710 def __exit__(self, exc_type, exc_value, exc_tb): |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
711 ... |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
712 |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
713 @abc.abstractmethod |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
714 def close(self): |
51921
f79f98733a5b
vfs: use @abstractmethod instead of homebrewing abstract methods
Matt Harbison <matt_harbison@yahoo.com>
parents:
51920
diff
changeset
|
715 ... |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
716 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
717 |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
718 class delayclosedfile(closewrapbase): |
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
719 """Proxy for a file object whose close is delayed. |
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
720 |
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
721 Do not instantiate outside of the vfs layer. |
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
722 """ |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
723 |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
724 def __init__(self, fh, closer): |
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
725 super(delayclosedfile, self).__init__(fh) |
43554
9f70512ae2cf
cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents:
43532
diff
changeset
|
726 object.__setattr__(self, '_closer', closer) |
29998
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
727 |
0c40e64d6154
scmutil: factor out common logic of delayclosedfile to reuse it
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29813
diff
changeset
|
728 def __exit__(self, exc_type, exc_value, exc_tb): |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
729 self._closer.close(self._origfh) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
730 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
731 def close(self): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
732 self._closer.close(self._origfh) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
733 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
734 |
49037
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48966
diff
changeset
|
735 class backgroundfilecloser: |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
736 """Coordinates background closing of file handles on multiple threads.""" |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
737 |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
738 def __init__(self, ui, expectedcount=-1): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
739 self._running = False |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
740 self._entered = False |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
741 self._threads = [] |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
742 self._threadexception = None |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
743 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
744 # Only Windows/NTFS has slow file closing. So only enable by default |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
745 # on that platform. But allow to be enabled elsewhere for testing. |
34645 | 746 defaultenabled = pycompat.iswindows |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
747 enabled = ui.configbool(b'worker', b'backgroundclose', defaultenabled) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
748 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
749 if not enabled: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
750 return |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
751 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
752 # There is overhead to starting and stopping the background threads. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
753 # Don't do background processing unless the file count is large enough |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
754 # to justify it. |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
755 minfilecount = ui.configint(b'worker', b'backgroundcloseminfilecount') |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
756 # FUTURE dynamically start background threads after minfilecount closes. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
757 # (We don't currently have any callers that don't know their file count) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
758 if expectedcount > 0 and expectedcount < minfilecount: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
759 return |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
760 |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
761 maxqueue = ui.configint(b'worker', b'backgroundclosemaxqueue') |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
762 threadcount = ui.configint(b'worker', b'backgroundclosethreadcount') |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
763 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
764 ui.debug( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
765 b'starting %d threads for background file closing\n' % threadcount |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
766 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
767 |
37890
8fb9985382be
pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35725
diff
changeset
|
768 self._queue = pycompat.queue.Queue(maxsize=maxqueue) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
769 self._running = True |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
770 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
771 for i in range(threadcount): |
43510
ffd632c224c3
vfs: another bytes-str confusion on thread name
Augie Fackler <augie@google.com>
parents:
43117
diff
changeset
|
772 t = threading.Thread(target=self._worker, name='backgroundcloser') |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
773 self._threads.append(t) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
774 t.start() |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
775 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
776 def __enter__(self): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
777 self._entered = True |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
778 return self |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
779 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
780 def __exit__(self, exc_type, exc_value, exc_tb): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
781 self._running = False |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
782 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
783 # Wait for threads to finish closing so open files don't linger for |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
784 # longer than lifetime of context manager. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
785 for t in self._threads: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
786 t.join() |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
787 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
788 def _worker(self): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
789 """Main routine for worker thread.""" |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
790 while True: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
791 try: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
792 fh = self._queue.get(block=True, timeout=0.100) |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
793 # Need to catch or the thread will terminate and |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
794 # we could orphan file descriptors. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
795 try: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
796 fh.close() |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
797 except Exception as e: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
798 # Stash so can re-raise from main thread later. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
799 self._threadexception = e |
37890
8fb9985382be
pycompat: export queue module instead of symbols in module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35725
diff
changeset
|
800 except pycompat.queue.Empty: |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
801 if not self._running: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
802 break |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
803 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
804 def close(self, fh): |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
805 """Schedule a file for closing.""" |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
806 if not self._entered: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
807 raise error.Abort( |
43117
8ff1ecfadcd1
cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents:
43090
diff
changeset
|
808 _(b'can only call close() when context manager active') |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
809 ) |
27895
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
810 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
811 # If a background thread encountered an exception, raise now so we fail |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
812 # fast. Otherwise we may potentially go on for minutes until the error |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
813 # is acted on. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
814 if self._threadexception: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
815 e = self._threadexception |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
816 self._threadexception = None |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
817 raise e |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
818 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
819 # If we're not actively running, close synchronously. |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
820 if not self._running: |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
821 fh.close() |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
822 return |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
823 |
2d6a89e79b48
scmutil: support background file closing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27879
diff
changeset
|
824 self._queue.put(fh, block=True, timeout=None) |
29999
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
825 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
826 |
29999
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
827 class checkambigatclosing(closewrapbase): |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
828 """Proxy for a file object, to avoid ambiguity of file stat |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
829 |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
830 See also util.filestat for detail about "ambiguity of file stat". |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
831 |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
832 This proxy is useful only if the target file is guarded by any |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
833 lock (e.g. repo.lock or repo.wlock) |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
834 |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
835 Do not instantiate outside of the vfs layer. |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
836 """ |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43027
diff
changeset
|
837 |
29999
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
838 def __init__(self, fh): |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
839 super(checkambigatclosing, self).__init__(fh) |
43554
9f70512ae2cf
cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents:
43532
diff
changeset
|
840 object.__setattr__(self, '_oldstat', util.filestat.frompath(fh.name)) |
29999
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
841 |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
842 def _checkambig(self): |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
843 oldstat = self._oldstat |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
844 if oldstat.stat: |
33280
646352291f5b
vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
33259
diff
changeset
|
845 _avoidambig(self._origfh.name, oldstat) |
29999
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
846 |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
847 def __exit__(self, exc_type, exc_value, exc_tb): |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
848 self._origfh.__exit__(exc_type, exc_value, exc_tb) |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
849 self._checkambig() |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
850 |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
851 def close(self): |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
852 self._origfh.close() |
57830bd0e787
scmutil: add file object wrapper class to check ambiguity at closing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29998
diff
changeset
|
853 self._checkambig() |