mercurial-scm/hg: mercurial/chgserver.py annotate

annotate mercurial/chgserver.py @ 45892:06b64fabf91c

copies: cache the ancestor checking call when tracing copy A good share of the time spent in this function is spent doing ancestors checking. To avoid spending time in duplicated call, we cache the result of calls. In the slower case, this provide a quite significant performance boost. Below are the result for a set of selected pairs (many of them pathological): (And further down is another table that summarize the current state of filelog based vs changeset base copy tracing) The benchmark have been configured to be killed after 6 minutes of runtime, which mean that any detect slower than 2 minutes will be marked as "killed". This drop some useful information about how much slower these case are? but also prevent 99% of the benchmark time to be spent on case that can be labelled "very slow" anyway. Repo Case Source-Rev Dest-Rev Old-Time New-Time Difference Factor ------------------------------------------------------------------------------------------------------------------------------------ mercurial x_revs_x_added_0_copies ad6b123de1c7 39cfcef4f463 : 0.000044 s, 0.000044 s, +0.000000 s, ? 1.0000 mercurial x_revs_x_added_x_copies 2b1c78674230 0c1d10351869 : 0.000138 s, 0.000138 s, +0.000000 s, ? 1.0000 mercurial x000_revs_x000_added_x_copies 81f8ff2a9bf2 dd3267698d84 : 0.005067 s, 0.005052 s, -0.000015 s, ? 0.9970 pypy x_revs_x_added_0_copies aed021ee8ae8 099ed31b181b : 0.000218 s, 0.000219 s, +0.000001 s, ? 1.0046 pypy x_revs_x000_added_0_copies 4aa4e1f8e19a 359343b9ac0e : 0.000053 s, 0.000055 s, +0.000002 s, ? 1.0377 pypy x_revs_x_added_x_copies ac52eb7bbbb0 72e022663155 : 0.000125 s, 0.000128 s, +0.000003 s, ? 1.0240 pypy x_revs_x00_added_x_copies c3b14617fbd7 ace7255d9a26 : 0.001098 s, 0.001089 s, -0.000009 s, ? 0.9918 pypy x_revs_x000_added_x000_copies df6f7a526b60 a83dc6a2d56f : 0.017546 s, 0.017407 s, -0.000139 s, ? 0.9921 pypy x000_revs_xx00_added_0_copies 89a76aede314 2f22446ff07e : 0.096723 s, 0.094175 s, -0.002548 s, ? 0.9737 pypy x000_revs_x000_added_x_copies 8a3b5bfd266e 2c68e87c3efe : 0.271796 s, 0.238009 s, -0.033787 s, ? 0.8757 pypy x000_revs_x000_added_x000_copies 89a76aede314 7b3dda341c84 : 0.128602 s, 0.125876 s, -0.002726 s, ? 0.9788 pypy x0000_revs_x_added_0_copies d1defd0dc478 c9cb1334cc78 : 7.086742 s, 3.581556 s, -3.505186 s, ? 0.5054 pypy x0000_revs_xx000_added_0_copies bf2c629d0071 4ffed77c095c : 0.016634 s, 0.016721 s, +0.000087 s, ? 1.0052 pypy x0000_revs_xx000_added_x000_copies 08ea3258278e d9fa043f30c0 : 0.254225 s, 0.242367 s, -0.011858 s, ? 0.9534 netbeans x_revs_x_added_0_copies fb0955ffcbcd a01e9239f9e7 : 0.000166 s, 0.000165 s, -0.000001 s, ? 0.9940 netbeans x_revs_x000_added_0_copies 6f360122949f 20eb231cc7d0 : 0.000118 s, 0.000114 s, -0.000004 s, ? 0.9661 netbeans x_revs_x_added_x_copies 1ada3faf6fb6 5a39d12eecf4 : 0.000296 s, 0.000296 s, +0.000000 s, ? 1.0000 netbeans x_revs_x00_added_x_copies 35be93ba1e2c 9eec5e90c05f : 0.001137 s, 0.001124 s, -0.000013 s, ? 0.9886 netbeans x000_revs_xx00_added_0_copies eac3045b4fdd 51d4ae7f1290 : 0.014133 s, 0.013060 s, -0.001073 s, ? 0.9241 netbeans x000_revs_x000_added_x_copies e2063d266acd 6081d72689dc : 0.016988 s, 0.017112 s, +0.000124 s, ? 1.0073 netbeans x000_revs_x000_added_x000_copies ff453e9fee32 411350406ec2 : 0.676361 s, 0.660350 s, -0.016011 s, ? 0.9763 netbeans x0000_revs_xx000_added_x000_copies 588c2d1ced70 1aad62e59ddd : 12.515149 s, 10.032499 s, -2.482650 s, ? 0.8016 mozilla-central x_revs_x_added_0_copies 3697f962bb7b 7015fcdd43a2 : 0.000186 s, 0.000189 s, +0.000003 s, ? 1.0161 mozilla-central x_revs_x000_added_0_copies dd390860c6c9 40d0c5bed75d : 0.000459 s, 0.000462 s, +0.000003 s, ? 1.0065 mozilla-central x_revs_x_added_x_copies 8d198483ae3b 14207ffc2b2f : 0.000273 s, 0.000270 s, -0.000003 s, ? 0.9890 mozilla-central x_revs_x00_added_x_copies 98cbc58cc6bc 446a150332c3 : 0.001503 s, 0.001474 s, -0.000029 s, ? 0.9807 mozilla-central x_revs_x000_added_x000_copies 3c684b4b8f68 0a5e72d1b479 : 0.004862 s, 0.004806 s, -0.000056 s, ? 0.9885 mozilla-central x_revs_x0000_added_x0000_copies effb563bb7e5 c07a39dc4e80 : 0.088291 s, 0.085150 s, -0.003141 s, ? 0.9644 mozilla-central x000_revs_xx00_added_0_copies 6100d773079a 04a55431795e : 0.007113 s, 0.007064 s, -0.000049 s, ? 0.9931 mozilla-central x000_revs_x000_added_x_copies 9f17a6fc04f9 2d37b966abed : 0.004687 s, 0.004741 s, +0.000054 s, ? 1.0115 mozilla-central x000_revs_x000_added_x000_copies 7c97034feb78 4407bd0c6330 : 0.198710 s, 0.190133 s, -0.008577 s, ? 0.9568 mozilla-central x0000_revs_xx000_added_0_copies 9eec5917337d 67118cc6dcad : 0.036068 s, 0.035651 s, -0.000417 s, ? 0.9884 mozilla-central x0000_revs_xx000_added_x000_copies f78c615a656c 96a38b690156 : 0.465362 s, 0.440694 s, -0.024668 s, ? 0.9470 mozilla-central x00000_revs_x0000_added_x0000_copies 6832ae71433c 4c222a1d9a00 : 24.519684 s, 18.454163 s, -6.065521 s, ? 0.7526 mozilla-central x00000_revs_x00000_added_x000_copies 76caed42cf7c 1daa622bbe42 : 42.711897 s, 31.562719 s, -11.149178 s, ? 0.7390 mozilla-try x_revs_x_added_0_copies aaf6dde0deb8 9790f499805a : 0.001201 s, 0.001189 s, -0.000012 s, ? 0.9900 mozilla-try x_revs_x000_added_0_copies d8d0222927b4 5bb8ce8c7450 : 0.001216 s, 0.001204 s, -0.000012 s, ? 0.9901 mozilla-try x_revs_x_added_x_copies 092fcca11bdb 936255a0384a : 0.000595 s, 0.000586 s, -0.000009 s, ? 0.9849 mozilla-try x_revs_x00_added_x_copies b53d2fadbdb5 017afae788ec : 0.001856 s, 0.001845 s, -0.000011 s, ? 0.9941 mozilla-try x_revs_x000_added_x000_copies 20408ad61ce5 6f0ee96e21ad : 0.064936 s, 0.063822 s, -0.001114 s, ? 0.9828 mozilla-try x_revs_x0000_added_x0000_copies effb563bb7e5 c07a39dc4e80 : 0.090601 s, 0.088038 s, -0.002563 s, ? 0.9717 mozilla-try x000_revs_xx00_added_0_copies 6100d773079a 04a55431795e : 0.007510 s, 0.007389 s, -0.000121 s, ? 0.9839 mozilla-try x000_revs_x000_added_x_copies 9f17a6fc04f9 2d37b966abed : 0.004911 s, 0.004868 s, -0.000043 s, ? 0.9912 mozilla-try x000_revs_x000_added_x000_copies 1346fd0130e4 4c65cbdabc1f : 0.233231 s, 0.222450 s, -0.010781 s, ? 0.9538 mozilla-try x0000_revs_x_added_0_copies 63519bfd42ee a36a2a865d92 : 0.419989 s, 0.370675 s, -0.049314 s, ? 0.8826 mozilla-try x0000_revs_x_added_x_copies 9fe69ff0762d bcabf2a78927 : 0.401521 s, 0.358020 s, -0.043501 s, ? 0.8917 mozilla-try x0000_revs_xx000_added_x_copies 156f6e2674f2 4d0f2c178e66 : 0.179555 s, 0.145235 s, -0.034320 s, ? 0.8089 mozilla-try x0000_revs_xx000_added_0_copies 9eec5917337d 67118cc6dcad : 0.038004 s, 0.037606 s, -0.000398 s, ? 0.9895 mozilla-try x0000_revs_xx000_added_x000_copies 89294cd501d9 7ccb2fc7ccb5 : 52.838482 s, 7.382439 s, -45.456043 s, ? 0.1397 mozilla-try x0000_revs_x0000_added_x0000_copies e928c65095ed e951f4ad123a : 8.705874 s, 7.273506 s, -1.432368 s, ? 0.8355 mozilla-try x00000_revs_x00000_added_0_copies dc8a3ca7010e d16fde900c9c : 1.126708 s, 1.074593 s, -0.052115 s, ? 0.9537 mozilla-try x00000_revs_x0000_added_x0000_copies 8d3fafa80d4b eb884023b810 : 83.854020 s, 27.746195 s, -56.107825 s, ? 0.3309 Below is a table comparing the runtime of the current "filelog centric" algorithm, with the "changeset centric" one, we just modified. The changeset centric algorithm is a significant win in many scenario, but they are still various cases where it is quite slower. When many revision has to be considered the cost of retrieving the copy information, creating new dictionaries, merging dictionaries and checking if revision are ancestors of each other can slow things down. The rest of this series, will introduce a rust version of the copy tracing code to deal with most of theses issues. Repo Case Source-Rev Dest-Rev filelog sidedata Difference Factor --------------------------------------------------------------------------------------------------------------------------------------- mercurial x_revs_x_added_0_copies ad6b123de1c7 39cfcef4f463 : 0.000914 s, 0.000044 s, - 0.000870 s, ? 0.048140 mercurial x_revs_x_added_x_copies 2b1c78674230 0c1d10351869 : 0.001812 s, 0.000138 s, - 0.001674 s, ? 0.076159 mercurial x000_revs_x000_added_x_copies 81f8ff2a9bf2 dd3267698d84 : 0.017954 s, 0.005052 s, - 0.012902 s, ? 0.281386 pypy x_revs_x_added_0_copies aed021ee8ae8 099ed31b181b : 0.001509 s, 0.000219 s, - 0.001290 s, ? 0.145129 pypy x_revs_x000_added_0_copies 4aa4e1f8e19a 359343b9ac0e : 0.206881 s, 0.000055 s, - 0.206826 s, ? 0.000266 pypy x_revs_x_added_x_copies ac52eb7bbbb0 72e022663155 : 0.016951 s, 0.000128 s, - 0.016823 s, ? 0.007551 pypy x_revs_x00_added_x_copies c3b14617fbd7 ace7255d9a26 : 0.019096 s, 0.001089 s, - 0.018007 s, ? 0.057028 pypy x_revs_x000_added_x000_copies df6f7a526b60 a83dc6a2d56f : 0.762506 s, 0.017407 s, - 0.745099 s, ? 0.022829 pypy x000_revs_xx00_added_0_copies 89a76aede314 2f22446ff07e : 1.179211 s, 0.094175 s, - 1.085036 s, ? 0.079863 pypy x000_revs_x000_added_x_copies 8a3b5bfd266e 2c68e87c3efe : 1.249058 s, 0.238009 s, - 1.011049 s, ? 0.190551 pypy x000_revs_x000_added_x000_copies 89a76aede314 7b3dda341c84 : 1.614107 s, 0.125876 s, - 1.488231 s, ? 0.077985 pypy x0000_revs_x_added_0_copies d1defd0dc478 c9cb1334cc78 : 0.001064 s, 3.581556 s, + 3.580492 s, ? 3366.124060 pypy x0000_revs_xx000_added_0_copies bf2c629d0071 4ffed77c095c : 1.061275 s, 0.016721 s, - 1.044554 s, ? 0.015756 pypy x0000_revs_xx000_added_x000_copies 08ea3258278e d9fa043f30c0 : 1.341119 s, 0.242367 s, - 1.098752 s, ? 0.180720 netbeans x_revs_x_added_0_copies fb0955ffcbcd a01e9239f9e7 : 0.027803 s, 0.000165 s, - 0.027638 s, ? 0.005935 netbeans x_revs_x000_added_0_copies 6f360122949f 20eb231cc7d0 : 0.130014 s, 0.000114 s, - 0.129900 s, ? 0.000877 netbeans x_revs_x_added_x_copies 1ada3faf6fb6 5a39d12eecf4 : 0.024990 s, 0.000296 s, - 0.024694 s, ? 0.011845 netbeans x_revs_x00_added_x_copies 35be93ba1e2c 9eec5e90c05f : 0.052201 s, 0.001124 s, - 0.051077 s, ? 0.021532 netbeans x000_revs_xx00_added_0_copies eac3045b4fdd 51d4ae7f1290 : 0.037642 s, 0.013060 s, - 0.024582 s, ? 0.346953 netbeans x000_revs_x000_added_x_copies e2063d266acd 6081d72689dc : 0.197086 s, 0.017112 s, - 0.179974 s, ? 0.086825 netbeans x000_revs_x000_added_x000_copies ff453e9fee32 411350406ec2 : 0.935148 s, 0.660350 s, - 0.274798 s, ? 0.706145 netbeans x0000_revs_xx000_added_x000_copies 588c2d1ced70 1aad62e59ddd : 3.920674 s, 10.032499 s, + 6.111825 s, ? 2.558871 mozilla-central x_revs_x_added_0_copies 3697f962bb7b 7015fcdd43a2 : 0.024232 s, 0.000189 s, - 0.024043 s, ? 0.007800 mozilla-central x_revs_x000_added_0_copies dd390860c6c9 40d0c5bed75d : 0.141483 s, 0.000462 s, - 0.141021 s, ? 0.003265 mozilla-central x_revs_x_added_x_copies 8d198483ae3b 14207ffc2b2f : 0.025775 s, 0.000270 s, - 0.025505 s, ? 0.010475 mozilla-central x_revs_x00_added_x_copies 98cbc58cc6bc 446a150332c3 : 0.084922 s, 0.001474 s, - 0.083448 s, ? 0.017357 mozilla-central x_revs_x000_added_x000_copies 3c684b4b8f68 0a5e72d1b479 : 0.194784 s, 0.004806 s, - 0.189978 s, ? 0.024673 mozilla-central x_revs_x0000_added_x0000_copies effb563bb7e5 c07a39dc4e80 : 2.161103 s, 0.085150 s, - 2.075953 s, ? 0.039401 mozilla-central x000_revs_xx00_added_0_copies 6100d773079a 04a55431795e : 0.089347 s, 0.007064 s, - 0.082283 s, ? 0.079063 mozilla-central x000_revs_x000_added_x_copies 9f17a6fc04f9 2d37b966abed : 0.732171 s, 0.004741 s, - 0.727430 s, ? 0.006475 mozilla-central x000_revs_x000_added_x000_copies 7c97034feb78 4407bd0c6330 : 1.157287 s, 0.190133 s, - 0.967154 s, ? 0.164292 mozilla-central x0000_revs_xx000_added_0_copies 9eec5917337d 67118cc6dcad : 6.726568 s, 0.035651 s, - 6.690917 s, ? 0.005300 mozilla-central x0000_revs_xx000_added_x000_copies f78c615a656c 96a38b690156 : 3.266229 s, 0.440694 s, - 2.825535 s, ? 0.134924 mozilla-central x00000_revs_x0000_added_x0000_copies 6832ae71433c 4c222a1d9a00 : 15.860534 s, 18.454163 s, + 2.593629 s, ? 1.163527 mozilla-central x00000_revs_x00000_added_x000_copies 76caed42cf7c 1daa622bbe42 : 20.450475 s, 31.562719 s, +11.112244 s, ? 1.543373 mozilla-try x_revs_x_added_0_copies aaf6dde0deb8 9790f499805a : 0.080442 s, 0.001189 s, - 0.079253 s, ? 0.014781 mozilla-try x_revs_x000_added_0_copies d8d0222927b4 5bb8ce8c7450 : 0.497672 s, 0.001204 s, - 0.496468 s, ? 0.002419 mozilla-try x_revs_x_added_x_copies 092fcca11bdb 936255a0384a : 0.021183 s, 0.000586 s, - 0.020597 s, ? 0.027664 mozilla-try x_revs_x00_added_x_copies b53d2fadbdb5 017afae788ec : 0.230991 s, 0.001845 s, - 0.229146 s, ? 0.007987 mozilla-try x_revs_x000_added_x000_copies 20408ad61ce5 6f0ee96e21ad : 1.118461 s, 0.063822 s, - 1.054639 s, ? 0.057062 mozilla-try x_revs_x0000_added_x0000_copies effb563bb7e5 c07a39dc4e80 : 2.206083 s, 0.088038 s, - 2.118045 s, ? 0.039907 mozilla-try x000_revs_xx00_added_0_copies 6100d773079a 04a55431795e : 0.089404 s, 0.007389 s, - 0.082015 s, ? 0.082647 mozilla-try x000_revs_x000_added_x_copies 9f17a6fc04f9 2d37b966abed : 0.733043 s, 0.004868 s, - 0.728175 s, ? 0.006641 mozilla-try x000_revs_x000_added_x000_copies 1346fd0130e4 4c65cbdabc1f : 1.163367 s, 0.222450 s, - 0.940917 s, ? 0.191212 mozilla-try x0000_revs_x_added_0_copies 63519bfd42ee a36a2a865d92 : 0.085456 s, 0.370675 s, + 0.285219 s, ? 4.337612 mozilla-try x0000_revs_x_added_x_copies 9fe69ff0762d bcabf2a78927 : 0.083601 s, 0.358020 s, + 0.274419 s, ? 4.282485 mozilla-try x0000_revs_xx000_added_x_copies 156f6e2674f2 4d0f2c178e66 : 7.366614 s, 0.145235 s, - 7.221379 s, ? 0.019715 mozilla-try x0000_revs_xx000_added_0_copies 9eec5917337d 67118cc6dcad : 6.664464 s, 0.037606 s, - 6.626858 s, ? 0.005643 mozilla-try x0000_revs_xx000_added_x000_copies 89294cd501d9 7ccb2fc7ccb5 : 7.467836 s, 7.382439 s, - 0.085397 s, ? 0.988565 mozilla-try x0000_revs_x0000_added_x0000_copies e928c65095ed e951f4ad123a : 9.801294 s, 7.273506 s, - 2.527788 s, ? 0.742097 mozilla-try x00000_revs_x_added_0_copies 6a320851d377 1ebb79acd503 : 0.091886 s, killed mozilla-try x00000_revs_x00000_added_0_copies dc8a3ca7010e d16fde900c9c : 26.491140 s, 1.074593 s, -25.416547 s, ? 0.040564 mozilla-try x00000_revs_x_added_x_copies 5173c4b6f97c 95d83ee7242d : 0.092863 s, killed mozilla-try x00000_revs_x000_added_x_copies 9126823d0e9c ca82787bb23c : 0.226823 s, killed mozilla-try x00000_revs_x0000_added_x0000_copies 8d3fafa80d4b eb884023b810 : 18.914630 s, 27.746195 s, + 8.831565 s, ? 1.466917 mozilla-try x00000_revs_x00000_added_x0000_copies 1b661134e2ca 1ae03d022d6d : 21.198903 s, killed mozilla-try x00000_revs_x00000_added_x000_copies 9b2a99adc05e 8e29777b48e6 : 24.952268 s, killed Differential Revision: https://phab.mercurial-scm.org/D9296

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Mon, 02 Nov 2020 11:03:56 +0100
parents	7eb221b9af6c
children	aab70f05d6ec

rev	line source
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	1 # chgserver.py - command server extension for cHg
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	2 #
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	3 # Copyright 2011 Yuya Nishihara <yuya@tcha.org>
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	4 #
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	7
30513 ff7df4bb75de chgserver: make it a core module and drop extension flags Yuya Nishihara <yuya@tcha.org> parents: 30512 diff changeset	8 """command server extension for cHg
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	9
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	10 'S' channel (read/write)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	11 propagate ui.system() request to client
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	12
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	13 'attachio' command
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	14 attach client's stdio passed by sendmsg()
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	15
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	16 'chdir' command
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	17 change current directory
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	18
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	19 'setenv' command
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	20 replace os.environ completely
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	21
40108 e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	22 'setumask' command (DEPRECATED)
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	23 'setumask2' command
28325 9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	24 set umask
9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	25
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	26 'validate' command
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	27 reload the config and check if the server is up to date
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	28
28325 9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	29 Config
9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	30 ------
9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	31
9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	32 ::
9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	33
9fec3cb8d128 chgserver: update docs Jun Wu <quark@fb.com> parents: 28277 diff changeset	34 [chgserver]
30990 cb899ee133d8 chgserver: move comments in config example Jun Wu <quark@fb.com> parents: 30924 diff changeset	35 # how long (in seconds) should an idle chg server exit
cb899ee133d8 chgserver: move comments in config example Jun Wu <quark@fb.com> parents: 30924 diff changeset	36 idletimeout = 3600
cb899ee133d8 chgserver: move comments in config example Jun Wu <quark@fb.com> parents: 30924 diff changeset	37
cb899ee133d8 chgserver: move comments in config example Jun Wu <quark@fb.com> parents: 30924 diff changeset	38 # whether to skip config or env change checks
cb899ee133d8 chgserver: move comments in config example Jun Wu <quark@fb.com> parents: 30924 diff changeset	39 skiphash = False
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	40 """
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	41
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	42 from __future__ import absolute_import
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	43
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	44 import inspect
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	45 import os
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	46 import re
32236 c8b9943c07eb commandserver: move "listen" responsibility from service to handler Jun Wu <quark@fb.com> parents: 32208 diff changeset	47 import socket
36781 ffa3026d4196 cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime Augie Fackler <augie@google.com> parents: 35582 diff changeset	48 import stat
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	49 import struct
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	50 import time
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	51
30513 ff7df4bb75de chgserver: make it a core module and drop extension flags Yuya Nishihara <yuya@tcha.org> parents: 30512 diff changeset	52 from .i18n import _
43089 c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	53 from .pycompat import (
c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	54 getattr,
c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	55 setattr,
c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	56 )
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	57
30513 ff7df4bb75de chgserver: make it a core module and drop extension flags Yuya Nishihara <yuya@tcha.org> parents: 30512 diff changeset	58 from . import (
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	59 commandserver,
30635 a150173da1c1 py3: replace os.environ with encoding.environ (part 2 of 5) Pulkit Goyal <7895pulkit@gmail.com> parents: 30619 diff changeset	60 encoding,
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	61 error,
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	62 extensions,
35582 72b91f905065 py3: use node.hex(h.digest()) instead of h.hexdigest() Pulkit Goyal <7895pulkit@gmail.com> parents: 35224 diff changeset	63 node,
30669 10b17ed9b591 py3: replace sys.executable with pycompat.sysexecutable Pulkit Goyal <7895pulkit@gmail.com> parents: 30645 diff changeset	64 pycompat,
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	65 util,
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	66 )
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	67
37119 d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	68 from .utils import (
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43860 diff changeset	69 hashutil,
37119 d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	70 procutil,
41828 bce2356ece68 py3: port things from chgserver.py Pulkit Goyal <pulkit@yandex-team.ru> parents: 41816 diff changeset	71 stringutil,
37119 d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	72 )
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	73
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	74
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	75 def _hashlist(items):
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	76 """return sha1 hexdigest for a list"""
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43860 diff changeset	77 return node.hex(hashutil.sha1(stringutil.pprint(items)).digest())
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	78
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	79
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	80 # sensitive config sections affecting confighash
28478 e6e183687545 chgserver: include [extdiff] in confighash Jun Wu <quark@fb.com> parents: 28454 diff changeset	81 _configsections = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	82 b'alias', # affects global state commands.table
44616 bdc8a5944d44 chgserver: add merge-tools to sensitive config items Pulkit Goyal <7895pulkit@gmail.com> parents: 44612 diff changeset	83 b'diff-tools', # affects whether gui or not in extdiff's uisetup
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	84 b'eol', # uses setconfig('eol', ...)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	85 b'extdiff', # uisetup will register new commands
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	86 b'extensions',
44612 bc592059d04f chgserver: add fastannotate config section to sensitive list Pulkit Goyal <7895pulkit@gmail.com> parents: 44359 diff changeset	87 b'fastannotate', # affects annotate command and adds fastannonate cmd
44616 bdc8a5944d44 chgserver: add merge-tools to sensitive config items Pulkit Goyal <7895pulkit@gmail.com> parents: 44612 diff changeset	88 b'merge-tools', # affects whether gui or not in extdiff's uisetup
44325 5cd2d91eeebd chgserver: spawn new process if schemes change Yuya Nishihara <yuya@tcha.org> parents: 44060 diff changeset	89 b'schemes', # extsetup will update global hg.schemes
28478 e6e183687545 chgserver: include [extdiff] in confighash Jun Wu <quark@fb.com> parents: 28454 diff changeset	90 ]
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	91
34839 110040e715c9 test-show: make it compatible with chg Jun Wu <quark@fb.com> parents: 34485 diff changeset	92 _configsectionitems = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	93 (b'commands', b'show.aliasprefix'), # show.py reads it in extsetup
34839 110040e715c9 test-show: make it compatible with chg Jun Wu <quark@fb.com> parents: 34485 diff changeset	94 ]
110040e715c9 test-show: make it compatible with chg Jun Wu <quark@fb.com> parents: 34485 diff changeset	95
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	96 # sensitive environment variables affecting confighash
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	97 _envre = re.compile(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	98 br'''\A(?:
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	99 CHGHG
32271 6096d27dc119 chgserver: more explicit about sensitive environ variables Jun Wu <quark@fb.com> parents: 32237 diff changeset	100 \|HG(?:DEMANDIMPORT\|EMITWARNINGS\|MODULEPOLICY\|PROF\|RCPATH)?
6096d27dc119 chgserver: more explicit about sensitive environ variables Jun Wu <quark@fb.com> parents: 32237 diff changeset	101 \|HG(?:ENCODING\|PLAIN).*
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	102 \|LANG(?:UAGE)?
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	103 \|LC_.*
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	104 \|LD_.*
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	105 \|PATH
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	106 \|PYTHON.*
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	107 \|TERM(?:INFO)?
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	108 \|TZ
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	109 )\Z''',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	110 re.X,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	111 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	112
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	113
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	114 def _confighash(ui):
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	115 """return a quick hash for detecting config/env changes
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	116
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	117 confighash is the hash of sensitive config items and environment variables.
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	118
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	119 for chgserver, it is designed that once confighash changes, the server is
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	120 not qualified to serve its client and should redirect the client to a new
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	121 server. different from mtimehash, confighash change will not mark the
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	122 server outdated and exit since the user can have different configs at the
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	123 same time.
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	124 """
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	125 sectionitems = []
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	126 for section in _configsections:
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	127 sectionitems.append(ui.configitems(section))
34839 110040e715c9 test-show: make it compatible with chg Jun Wu <quark@fb.com> parents: 34485 diff changeset	128 for section, item in _configsectionitems:
110040e715c9 test-show: make it compatible with chg Jun Wu <quark@fb.com> parents: 34485 diff changeset	129 sectionitems.append(ui.config(section, item))
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	130 sectionhash = _hashlist(sectionitems)
34887 7bf7544fd6cc chgserver: do not treat HG as sensitive environ when CHGHG is set Jun Wu <quark@fb.com> parents: 34839 diff changeset	131 # If $CHGHG is set, the change to $HG should not trigger a new chg server
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	132 if b'CHGHG' in encoding.environ:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	133 ignored = {b'HG'}
34887 7bf7544fd6cc chgserver: do not treat HG as sensitive environ when CHGHG is set Jun Wu <quark@fb.com> parents: 34839 diff changeset	134 else:
7bf7544fd6cc chgserver: do not treat HG as sensitive environ when CHGHG is set Jun Wu <quark@fb.com> parents: 34839 diff changeset	135 ignored = set()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	136 envitems = [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	137 (k, v)
43106 d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer Gregory Szorc <gregory.szorc@gmail.com> parents: 43089 diff changeset	138 for k, v in pycompat.iteritems(encoding.environ)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	139 if _envre.match(k) and k not in ignored
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	140 ]
28262 53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	141 envhash = _hashlist(sorted(envitems))
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	142 return sectionhash[:6] + envhash[:6]
53dc4aada2d9 chgserver: add utilities to calculate confighash Jun Wu <quark@fb.com> parents: 28261 diff changeset	143
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	144
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	145 def _getmtimepaths(ui):
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	146 """get a list of paths that should be checked to detect change
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	147
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	148 The list will include:
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	149 - extensions (will not cover all files for complex extensions)
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	150 - mercurial/__version__.py
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	151 - python binary
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	152 """
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	153 modules = [m for n, m in extensions.extensions(ui)]
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	154 try:
30513 ff7df4bb75de chgserver: make it a core module and drop extension flags Yuya Nishihara <yuya@tcha.org> parents: 30512 diff changeset	155 from . import __version__
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	156
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	157 modules.append(__version__)
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	158 except ImportError:
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	159 pass
42523 49998d5ba66a pycompat: make fewer assumptions about sys.executable Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 41828 diff changeset	160 files = []
49998d5ba66a pycompat: make fewer assumptions about sys.executable Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 41828 diff changeset	161 if pycompat.sysexecutable:
49998d5ba66a pycompat: make fewer assumptions about sys.executable Rodrigo Damazio Bovendorp <rdamazio@google.com> parents: 41828 diff changeset	162 files.append(pycompat.sysexecutable)
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	163 for m in modules:
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	164 try:
41816 78027e7bc544 py3: convert return values of inspect.getabsfile() to bytes Pulkit Goyal <pulkit@yandex-team.ru> parents: 41285 diff changeset	165 files.append(pycompat.fsencode(inspect.getabsfile(m)))
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	166 except TypeError:
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	167 pass
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	168 return sorted(set(files))
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	169
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	170
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	171 def _mtimehash(paths):
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	172 """return a quick hash for detecting file changes
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	173
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	174 mtimehash calls stat on given paths and calculate a hash based on size and
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	175 mtime of each file. mtimehash does not read file content because reading is
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	176 expensive. therefore it's not 100% reliable for detecting content changes.
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	177 it's possible to return different hashes for same file contents.
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	178 it's also possible to return a same hash for different file contents for
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	179 some carefully crafted situation.
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	180
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	181 for chgserver, it is designed that once mtimehash changes, the server is
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	182 considered outdated immediately and should no longer provide service.
29462 71ed5a3ef8a9 chgserver: document why we don't merge mtimehash and confighash Jun Wu <quark@fb.com> parents: 29433 diff changeset	183
71ed5a3ef8a9 chgserver: document why we don't merge mtimehash and confighash Jun Wu <quark@fb.com> parents: 29433 diff changeset	184 mtimehash is not included in confighash because we only know the paths of
71ed5a3ef8a9 chgserver: document why we don't merge mtimehash and confighash Jun Wu <quark@fb.com> parents: 29433 diff changeset	185 extensions after importing them (there is imp.find_module but that faces
71ed5a3ef8a9 chgserver: document why we don't merge mtimehash and confighash Jun Wu <quark@fb.com> parents: 29433 diff changeset	186 race conditions). We need to calculate confighash without importing.
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	187 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	188
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	189 def trystat(path):
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	190 try:
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	191 st = os.stat(path)
36781 ffa3026d4196 cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime Augie Fackler <augie@google.com> parents: 35582 diff changeset	192 return (st[stat.ST_MTIME], st.st_size)
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	193 except OSError:
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	194 # could be ENOENT, EPERM etc. not fatal in any case
b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	195 pass
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	196
42818 0cbe17335857 py3: use pycompat.maplist() in chgserver Martin von Zweigbergk <martinvonz@google.com> parents: 42523 diff changeset	197 return _hashlist(pycompat.maplist(trystat, paths))[:12]
28276 b4ceadb2c439 chgserver: add utilities to calculate mtimehash Jun Wu <quark@fb.com> parents: 28264 diff changeset	198
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	199
28277 cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	200 class hashstate(object):
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	201 """a structure storing confighash, mtimehash, paths used for mtimehash"""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	202
28277 cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	203 def __init__(self, confighash, mtimehash, mtimepaths):
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	204 self.confighash = confighash
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	205 self.mtimehash = mtimehash
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	206 self.mtimepaths = mtimepaths
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	207
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	208 @staticmethod
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	209 def fromui(ui, mtimepaths=None):
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	210 if mtimepaths is None:
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	211 mtimepaths = _getmtimepaths(ui)
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	212 confighash = _confighash(ui)
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	213 mtimehash = _mtimehash(mtimepaths)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	214 ui.log(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	215 b'cmdserver',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	216 b'confighash = %s mtimehash = %s\n',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	217 confighash,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	218 mtimehash,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	219 )
28277 cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	220 return hashstate(confighash, mtimehash, mtimepaths)
cdc6319f6a7d chgserver: add a structure for confighash and mtimehash Jun Wu <quark@fb.com> parents: 28276 diff changeset	221
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	222
30740 493935e0327a chgserver: implement chgui._runpager Jun Wu <quark@fb.com> parents: 30739 diff changeset	223 def _newchgui(srcui, csystem, attachio):
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	224 class chgui(srcui.__class__):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	225 def __init__(self, src=None):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	226 super(chgui, self).__init__(src)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	227 if src:
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	228 self._csystem = getattr(src, '_csystem', csystem)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	229 else:
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	230 self._csystem = csystem
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	231
31108 3f8f53190d6a chg: deduplicate error handling of ui.system() Yuya Nishihara <yuya@tcha.org> parents: 31107 diff changeset	232 def _runsystem(self, cmd, environ, cwd, out):
39808 1b9be0b26511 chgserver: update comment describing when to fall back to core _runsystem() Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	233 # fallback to the original system method if
1b9be0b26511 chgserver: update comment describing when to fall back to core _runsystem() Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	234 # a. the output stream is not stdout (e.g. stderr, cStringIO),
41285 cf8677cd7286 ui: proxy protect/restorestdio() calls to update internal flag Yuya Nishihara <yuya@tcha.org> parents: 40875 diff changeset	235 # b. or stdout is redirected by protectfinout(),
39808 1b9be0b26511 chgserver: update comment describing when to fall back to core _runsystem() Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	236 # because the chg client is not aware of these situations and
1b9be0b26511 chgserver: update comment describing when to fall back to core _runsystem() Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	237 # will behave differently (i.e. write to stdout).
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	238 if (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	239 out is not self.fout
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	240 or not util.safehasattr(self.fout, b'fileno')
39809 23a00bc90a3c chgserver: do not send system() back to client if stdio redirected (issue5992) Yuya Nishihara <yuya@tcha.org> parents: 39808 diff changeset	241 or self.fout.fileno() != procutil.stdout.fileno()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	242 or self._finoutredirected
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	243 ):
37120 a8a902d7176e procutil: bulk-replace function calls to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37119 diff changeset	244 return procutil.system(cmd, environ=environ, cwd=cwd, out=out)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	245 self.flush()
37120 a8a902d7176e procutil: bulk-replace function calls to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37119 diff changeset	246 return self._csystem(cmd, procutil.shellenviron(environ), cwd)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	247
31954 e518192d6bac pager: set some environment variables if they're not set Jun Wu <quark@fb.com> parents: 31695 diff changeset	248 def _runpager(self, cmd, env=None):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	249 self._csystem(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	250 cmd,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	251 procutil.shellenviron(env),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	252 type=b'pager',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	253 cmdtable={b'attachio': attachio},
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	254 )
31690 2d11d278279a ui: defer setting pager related properties until the pager has spawned Matt Harbison <matt_harbison@yahoo.com> parents: 31545 diff changeset	255 return True
30740 493935e0327a chgserver: implement chgui._runpager Jun Wu <quark@fb.com> parents: 30739 diff changeset	256
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	257 return chgui(srcui)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	258
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	259
40824 82210d88d814 commandserver: install logger to record server events through canonical API Yuya Nishihara <yuya@tcha.org> parents: 40729 diff changeset	260 def _loadnewui(srcui, args, cdebug):
30513 ff7df4bb75de chgserver: make it a core module and drop extension flags Yuya Nishihara <yuya@tcha.org> parents: 30512 diff changeset	261 from . import dispatch # avoid cycle
30512 cc374292a561 chgserver: delay importing commands and dispatch modules Yuya Nishihara <yuya@tcha.org> parents: 30511 diff changeset	262
30572 cfb227016d01 chgserver: call "load" for new ui objects Jun Wu <quark@fb.com> parents: 30513 diff changeset	263 newui = srcui.__class__.load()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	264 for a in [b'fin', b'fout', b'ferr', b'environ']:
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	265 setattr(newui, a, getattr(srcui, a))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	266 if util.safehasattr(srcui, b'_csystem'):
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	267 newui._csystem = srcui._csystem
28264 3682e201cce6 chgserver: make _renewui load repo and command line configs Jun Wu <quark@fb.com> parents: 28262 diff changeset	268
3682e201cce6 chgserver: make _renewui load repo and command line configs Jun Wu <quark@fb.com> parents: 28262 diff changeset	269 # command line args
35224 6e6d0a5b88e6 dispatch: replace _earlyreq() with new fancyopts-based parser Yuya Nishihara <yuya@tcha.org>* parents: 35170 diff changeset	270 options = dispatch._earlyparseopts(newui, args)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	271 dispatch._parseconfig(newui, options[b'config'])
28264 3682e201cce6 chgserver: make _renewui load repo and command line configs Jun Wu <quark@fb.com> parents: 28262 diff changeset	272
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	273 # stolen from tortoisehg.util.copydynamicconfig()
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	274 for section, name, value in srcui.walkconfig():
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	275 source = srcui.configsource(section, name)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	276 if b':' in source or source == b'--config' or source.startswith(b'$'):
31695 d73490957d61 chgserver: do not copy configs set by environment variables Jun Wu <quark@fb.com> parents: 31690 diff changeset	277 # path:line or command line, or environ
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	278 continue
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	279 newui.setconfig(section, name, value, source)
28599 0e7a929754aa chgserver: use global ui instead of repo ui for dispatch.request.ui Jun Wu <quark@fb.com> parents: 28586 diff changeset	280
0e7a929754aa chgserver: use global ui instead of repo ui for dispatch.request.ui Jun Wu <quark@fb.com> parents: 28586 diff changeset	281 # load wd and repo config, copied from dispatch.py
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	282 cwd = options[b'cwd']
35170 c9740b69b9b7 dispatch: add HGPLAIN=+strictflags to restrict early parsing of global options Yuya Nishihara <yuya@tcha.org> parents: 35033 diff changeset	283 cwd = cwd and os.path.realpath(cwd) or None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	284 rpath = options[b'repository']
28599 0e7a929754aa chgserver: use global ui instead of repo ui for dispatch.request.ui Jun Wu <quark@fb.com> parents: 28586 diff changeset	285 path, newlui = dispatch._getlocal(newui, rpath, wd=cwd)
0e7a929754aa chgserver: use global ui instead of repo ui for dispatch.request.ui Jun Wu <quark@fb.com> parents: 28586 diff changeset	286
40729 c93d046d4300 extensions: add "uipopulate" hook, called per instance, not per process Yuya Nishihara <yuya@tcha.org> parents: 40110 diff changeset	287 extensions.populateui(newui)
40824 82210d88d814 commandserver: install logger to record server events through canonical API Yuya Nishihara <yuya@tcha.org> parents: 40729 diff changeset	288 commandserver.setuplogging(newui, fp=cdebug)
40729 c93d046d4300 extensions: add "uipopulate" hook, called per instance, not per process Yuya Nishihara <yuya@tcha.org> parents: 40110 diff changeset	289 if newui is not newlui:
c93d046d4300 extensions: add "uipopulate" hook, called per instance, not per process Yuya Nishihara <yuya@tcha.org> parents: 40110 diff changeset	290 extensions.populateui(newlui)
40824 82210d88d814 commandserver: install logger to record server events through canonical API Yuya Nishihara <yuya@tcha.org> parents: 40729 diff changeset	291 commandserver.setuplogging(newlui, fp=cdebug)
40729 c93d046d4300 extensions: add "uipopulate" hook, called per instance, not per process Yuya Nishihara <yuya@tcha.org> parents: 40110 diff changeset	292
28599 0e7a929754aa chgserver: use global ui instead of repo ui for dispatch.request.ui Jun Wu <quark@fb.com> parents: 28586 diff changeset	293 return (newui, newlui)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	294
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	295
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	296 class channeledsystem(object):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	297 """Propagate ui.system() request in the following format:
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	298
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	299 payload length (unsigned int),
30726 dd897eb1699e chg: send type information via S channel (BC) Jun Wu <quark@fb.com> parents: 30669 diff changeset	300 type, '\0',
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	301 cmd, '\0',
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	302 cwd, '\0',
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	303 envkey, '=', val, '\0',
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	304 ...
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	305 envkey, '=', val
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	306
30726 dd897eb1699e chg: send type information via S channel (BC) Jun Wu <quark@fb.com> parents: 30669 diff changeset	307 if type == 'system', waits for:
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	308
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	309 exitcode length (unsigned int),
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	310 exitcode (int)
30739 815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	311
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	312 if type == 'pager', repetitively waits for a command name ending with '\n'
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	313 and executes it defined by cmdtable, or exits the loop if the command name
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	314 is empty.
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	315 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	316
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	317 def __init__(self, in_, out, channel):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	318 self.in_ = in_
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	319 self.out = out
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	320 self.channel = channel
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	321
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	322 def __call__(self, cmd, environ, cwd=None, type=b'system', cmdtable=None):
44867 8e8fd938ca07 cleanup: eliminate procutil.quotecommand() Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	323 args = [type, cmd, os.path.abspath(cwd or b'.')]
43106 d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer Gregory Szorc <gregory.szorc@gmail.com> parents: 43089 diff changeset	324 args.extend(b'%s=%s' % (k, v) for k, v in pycompat.iteritems(environ))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	325 data = b'\0'.join(args)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	326 self.out.write(struct.pack(b'>cI', self.channel, len(data)))
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	327 self.out.write(data)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	328 self.out.flush()
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	329
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	330 if type == b'system':
30727 18eb63ec8475 chgserver: check type passed to S channel Jun Wu <quark@fb.com> parents: 30726 diff changeset	331 length = self.in_.read(4)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	332 (length,) = struct.unpack(b'>I', length)
30727 18eb63ec8475 chgserver: check type passed to S channel Jun Wu <quark@fb.com> parents: 30726 diff changeset	333 if length != 4:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	334 raise error.Abort(_(b'invalid response'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	335 (rc,) = struct.unpack(b'>i', self.in_.read(4))
30727 18eb63ec8475 chgserver: check type passed to S channel Jun Wu <quark@fb.com> parents: 30726 diff changeset	336 return rc
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	337 elif type == b'pager':
30739 815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	338 while True:
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	339 cmd = self.in_.readline()[:-1]
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	340 if not cmd:
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	341 break
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	342 if cmdtable and cmd in cmdtable:
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	343 cmdtable[cmd]()
815e1cefd082 chgserver: make S channel support pager request Jun Wu <quark@fb.com> parents: 30737 diff changeset	344 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	345 raise error.Abort(_(b'unexpected command: %s') % cmd)
30727 18eb63ec8475 chgserver: check type passed to S channel Jun Wu <quark@fb.com> parents: 30726 diff changeset	346 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	347 raise error.ProgrammingError(b'invalid S channel type: %s' % type)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	348
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	349
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	350 _iochannels = [
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	351 # server.ch, ui.fp, mode
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43106 diff changeset	352 (b'cin', b'fin', 'rb'),
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43106 diff changeset	353 (b'cout', b'fout', 'wb'),
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43106 diff changeset	354 (b'cerr', b'ferr', 'wb'),
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	355 ]
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	356
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	357
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	358 class chgcmdserver(commandserver.server):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	359 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	360 self, ui, repo, fin, fout, sock, prereposetups, hashstate, baseaddress
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	361 ):
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	362 super(chgcmdserver, self).__init__(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	363 _newchgui(ui, channeledsystem(fin, fout, b'S'), self.attachio),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	364 repo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	365 fin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	366 fout,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	367 prereposetups,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	368 )
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	369 self.clientsock = sock
39738 a93fe297dfb3 chgserver: add separate flag to remember if stdio fds are replaced Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	370 self._ioattached = False
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	371 self._oldios = [] # original (self.ch, ui.fp, fd) before "attachio"
28328 e00e57d83653 chgserver: pass hashstate and base server address to chgcmdserver Jun Wu <quark@fb.com> parents: 28327 diff changeset	372 self.hashstate = hashstate
e00e57d83653 chgserver: pass hashstate and base server address to chgcmdserver Jun Wu <quark@fb.com> parents: 28327 diff changeset	373 self.baseaddress = baseaddress
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	374 if hashstate is not None:
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	375 self.capabilities = self.capabilities.copy()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	376 self.capabilities[b'validate'] = chgcmdserver.validate
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	377
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	378 def cleanup(self):
29512 538d0003c9e0 commandserver: promote .cleanup() hook from chgserver Yuya Nishihara <yuya@tcha.org> parents: 29511 diff changeset	379 super(chgcmdserver, self).cleanup()
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	380 # dispatch._runcatch() does not flush outputs if exception is not
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	381 # handled by dispatch._dispatch()
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	382 self.ui.flush()
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	383 self._restoreio()
39738 a93fe297dfb3 chgserver: add separate flag to remember if stdio fds are replaced Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	384 self._ioattached = False
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	385
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	386 def attachio(self):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	387 """Attach to client's stdio passed via unix domain socket; all
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	388 channels except cresult will no longer be used
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	389 """
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	390 # tell client to sendmsg() with 1-byte payload, which makes it
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	391 # distinctive from "attachio\n" command consumed by client.read()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	392 self.clientsock.sendall(struct.pack(b'>cI', b'I', 1))
32208 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 31954 diff changeset	393 clientfds = util.recvfds(self.clientsock.fileno())
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	394 self.ui.log(b'chgserver', b'received fds: %r\n', clientfds)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	395
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	396 ui = self.ui
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	397 ui.flush()
39738 a93fe297dfb3 chgserver: add separate flag to remember if stdio fds are replaced Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	398 self._saveio()
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	399 for fd, (cn, fn, mode) in zip(clientfds, _iochannels):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	400 assert fd > 0
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	401 fp = getattr(ui, fn)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	402 os.dup2(fd, fp.fileno())
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	403 os.close(fd)
39738 a93fe297dfb3 chgserver: add separate flag to remember if stdio fds are replaced Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	404 if self._ioattached:
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	405 continue
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	406 # reset buffering mode when client is first attached. as we want
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	407 # to see output immediately on pager, the mode stays unchanged
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	408 # when client re-attached. ferr is unchanged because it should
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	409 # be unbuffered no matter if it is a tty or not.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	410 if fn == b'ferr':
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	411 newfp = fp
45852 b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	412 elif pycompat.ispy3:
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	413 # On Python 3, the standard library doesn't offer line-buffered
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	414 # binary streams, so wrap/unwrap it.
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	415 if fp.isatty():
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	416 newfp = procutil.make_line_buffered(fp)
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	417 else:
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	418 newfp = procutil.unwrap_line_buffered(fp)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	419 else:
45852 b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	420 # Python 2 uses the I/O streams provided by the C library, so
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	421 # make it line-buffered explicitly. Otherwise the default would
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	422 # be decided on first write(), where fout could be a pager.
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	423 if fp.isatty():
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	424 bufsize = 1 # line buffered
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	425 else:
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	426 bufsize = -1 # system default
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	427 newfp = os.fdopen(fp.fileno(), mode, bufsize)
45852 b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	428 if newfp is not fp:
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	429 setattr(ui, fn, newfp)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	430 setattr(self, cn, newfp)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	431
39738 a93fe297dfb3 chgserver: add separate flag to remember if stdio fds are replaced Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	432 self._ioattached = True
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	433 self.cresult.write(struct.pack(b'>i', len(clientfds)))
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	434
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	435 def _saveio(self):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	436 if self._oldios:
39738 a93fe297dfb3 chgserver: add separate flag to remember if stdio fds are replaced Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	437 return
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	438 ui = self.ui
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	439 for cn, fn, _mode in _iochannels:
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	440 ch = getattr(self, cn)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	441 fp = getattr(ui, fn)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	442 fd = os.dup(fp.fileno())
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	443 self._oldios.append((ch, fp, fd))
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	444
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	445 def _restoreio(self):
45185 a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	446 if not self._oldios:
a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	447 return
a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	448 nullfd = os.open(os.devnull, os.O_WRONLY)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	449 ui = self.ui
45185 a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	450 for (ch, fp, fd), (cn, fn, mode) in zip(self._oldios, _iochannels):
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	451 newfp = getattr(ui, fn)
45852 b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	452 # On Python 2, newfp and fp may be separate file objects associated
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	453 # with the same fd, so we must close newfp while it's associated
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	454 # with the client. Otherwise the new associated fd would be closed
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	455 # when newfp gets deleted. On Python 3, newfp is just a wrapper
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	456 # around fp even if newfp is not fp, so deleting newfp is safe.
b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	457 if not (pycompat.ispy3 or newfp is fp):
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	458 newfp.close()
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	459 # restore original fd: fp is open again
45035 3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	460 try:
45852 b56feaa9b520 chgserver: backport py3 buffered I/O workarounds from procutil Yuya Nishihara <yuya@tcha.org> parents: 45682 diff changeset	461 if (pycompat.ispy3 or newfp is fp) and 'w' in mode:
45185 a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	462 # Discard buffered data which couldn't be flushed because
a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	463 # of EPIPE. The data should belong to the current session
a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	464 # and should never persist.
a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	465 os.dup2(nullfd, fp.fileno())
a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	466 fp.flush()
45035 3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	467 os.dup2(fd, fp.fileno())
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	468 except OSError as err:
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	469 # According to issue6330, running chg on heavy loaded systems
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	470 # can lead to EBUSY. [man dup2] indicates that, on Linux,
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	471 # EBUSY comes from a race condition between open() and dup2().
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	472 # However it's not clear why open() race occurred for
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	473 # newfd=stdin/out/err.
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	474 self.ui.log(
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	475 b'chgserver',
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	476 b'got %s while duplicating %s\n',
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	477 stringutil.forcebytestr(err),
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	478 fn,
3862de62d5cf chg: suppress OSError in _restoreio() and add some logging (issue6330) Pulkit Goyal <7895pulkit@gmail.com> parents: 44867 diff changeset	479 )
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	480 os.close(fd)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	481 setattr(self, cn, ch)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	482 setattr(ui, fn, fp)
45185 a17454a189d1 chgserver: discard buffered output before restoring fds (issue6207) Yuya Nishihara <yuya@tcha.org> parents: 45035 diff changeset	483 os.close(nullfd)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	484 del self._oldios[:]
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	485
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	486 def validate(self):
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	487 """Reload the config and check if the server is up to date
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	488
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	489 Read a list of '\0' separated arguments.
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	490 Write a non-empty list of '\0' separated instruction strings or '\0'
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	491 if the list is empty.
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	492 An instruction string could be either:
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	493 - "unlink $path", the client should unlink the path to stop the
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	494 outdated server.
28535 aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	495 - "redirect $path", the client should attempt to connect to $path
aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	496 first. If it does not work, start a new server. It implies
aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	497 "reconnect".
28516 3bf2892f685f chgserver: handle ParseError during validate Jun Wu <quark@fb.com> parents: 28514 diff changeset	498 - "exit $n", the client should exit directly with code n.
3bf2892f685f chgserver: handle ParseError during validate Jun Wu <quark@fb.com> parents: 28514 diff changeset	499 This may happen if we cannot parse the config.
28535 aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	500 - "reconnect", the client should close the connection and
aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	501 reconnect.
aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	502 If neither "reconnect" nor "redirect" is included in the instruction
aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	503 list, the client can continue with this server after completing all
aa082a8125da chgserver: add an explicit "reconnect" instruction to validate Jun Wu <quark@fb.com> parents: 28516 diff changeset	504 the instructions.
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	505 """
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	506 args = self._readlist()
28516 3bf2892f685f chgserver: handle ParseError during validate Jun Wu <quark@fb.com> parents: 28514 diff changeset	507 try:
40824 82210d88d814 commandserver: install logger to record server events through canonical API Yuya Nishihara <yuya@tcha.org> parents: 40729 diff changeset	508 self.ui, lui = _loadnewui(self.ui, args, self.cdebug)
40110 d1338b4e39d0 chgserver: catch Abort while parsing early args to shut down cleanly Yuya Nishihara <yuya@tcha.org> parents: 40108 diff changeset	509 except error.Abort as inst:
45885 600aec73f309 errors: format "abort: " text in a new Abort.format() method Martin von Zweigbergk <martinvonz@google.com> parents: 45884 diff changeset	510 self.ui.error(inst.format())
40110 d1338b4e39d0 chgserver: catch Abort while parsing early args to shut down cleanly Yuya Nishihara <yuya@tcha.org> parents: 40108 diff changeset	511 self.ui.flush()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	512 self.cresult.write(b'exit 255')
40110 d1338b4e39d0 chgserver: catch Abort while parsing early args to shut down cleanly Yuya Nishihara <yuya@tcha.org> parents: 40108 diff changeset	513 return
28599 0e7a929754aa chgserver: use global ui instead of repo ui for dispatch.request.ui Jun Wu <quark@fb.com> parents: 28586 diff changeset	514 newhash = hashstate.fromui(lui, self.hashstate.mtimepaths)
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	515 insts = []
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	516 if newhash.mtimehash != self.hashstate.mtimehash:
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	517 addr = _hashaddress(self.baseaddress, self.hashstate.confighash)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	518 insts.append(b'unlink %s' % addr)
28536 a979f5b03320 chgserver: invalidate the server if extensions fail to load Jun Wu <quark@fb.com> parents: 28535 diff changeset	519 # mtimehash is empty if one or more extensions fail to load.
a979f5b03320 chgserver: invalidate the server if extensions fail to load Jun Wu <quark@fb.com> parents: 28535 diff changeset	520 # to be compatible with hg, still serve the client this time.
a979f5b03320 chgserver: invalidate the server if extensions fail to load Jun Wu <quark@fb.com> parents: 28535 diff changeset	521 if self.hashstate.mtimehash:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	522 insts.append(b'reconnect')
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	523 if newhash.confighash != self.hashstate.confighash:
8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	524 addr = _hashaddress(self.baseaddress, newhash.confighash)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	525 insts.append(b'redirect %s' % addr)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	526 self.ui.log(b'chgserver', b'validate: %s\n', stringutil.pprint(insts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	527 self.cresult.write(b'\0'.join(insts) or b'\0')
28350 8f9661d1637b chgserver: implement validate command Jun Wu <quark@fb.com> parents: 28342 diff changeset	528
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	529 def chdir(self):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	530 """Change current directory
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	531
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	532 Note that the behavior of --cwd option is bit different from this.
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	533 It does not affect --config parameter.
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	534 """
28158 7cc57a531f0c chgserver: use _readlist and _readstr Jun Wu <quark@fb.com> parents: 28014 diff changeset	535 path = self._readstr()
7cc57a531f0c chgserver: use _readlist and _readstr Jun Wu <quark@fb.com> parents: 28014 diff changeset	536 if not path:
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	537 return
43621 975e517451a6 py3: replace "%r" by"'%s'% for py3-compatible (and clearer) quoting in chg Martin von Zweigbergk <martinvonz@google.com> parents: 43506 diff changeset	538 self.ui.log(b'chgserver', b"chdir to '%s'\n", path)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	539 os.chdir(path)
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	540
28159 d2d04d1d2f92 chgserver: add setumask method Jun Wu <quark@fb.com> parents: 28158 diff changeset	541 def setumask(self):
40108 e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	542 """Change umask (DEPRECATED)"""
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	543 # BUG: this does not follow the message frame structure, but kept for
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	544 # backward compatibility with old chg clients for some time
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	545 self._setumask(self._read(4))
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	546
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	547 def setumask2(self):
28159 d2d04d1d2f92 chgserver: add setumask method Jun Wu <quark@fb.com> parents: 28158 diff changeset	548 """Change umask"""
40108 e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	549 data = self._readstr()
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	550 if len(data) != 4:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	551 raise ValueError(b'invalid mask length in setumask2 request')
40108 e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	552 self._setumask(data)
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	553
e5fbdc3659fc chgserver: add "setumask2" command which uses correct message frame Yuya Nishihara <yuya@tcha.org> parents: 39840 diff changeset	554 def _setumask(self, data):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	555 mask = struct.unpack(b'>I', data)[0]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	556 self.ui.log(b'chgserver', b'setumask %r\n', mask)
44629 d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44616 diff changeset	557 util.setumask(mask)
28159 d2d04d1d2f92 chgserver: add setumask method Jun Wu <quark@fb.com> parents: 28158 diff changeset	558
30644 2bb8c53be961 chgserver: override runcommand Jun Wu <quark@fb.com> parents: 30643 diff changeset	559 def runcommand(self):
39739 7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	560 # pager may be attached within the runcommand session, which should
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	561 # be detached at the end of the session. otherwise the pager wouldn't
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	562 # receive EOF.
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	563 globaloldios = self._oldios
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	564 self._oldios = []
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	565 try:
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	566 return super(chgcmdserver, self).runcommand()
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	567 finally:
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	568 self._restoreio()
7cdd47d9ccf8 chgserver: restore pager fds attached within runcommand session Yuya Nishihara <yuya@tcha.org> parents: 39738 diff changeset	569 self._oldios = globaloldios
30644 2bb8c53be961 chgserver: override runcommand Jun Wu <quark@fb.com> parents: 30643 diff changeset	570
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	571 def setenv(self):
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	572 """Clear and update os.environ
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	573
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	574 Note that not all variables can make an effect on the running process.
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	575 """
28158 7cc57a531f0c chgserver: use _readlist and _readstr Jun Wu <quark@fb.com> parents: 28014 diff changeset	576 l = self._readlist()
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	577 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	578 newenv = dict(s.split(b'=', 1) for s in l)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	579 except ValueError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	580 raise ValueError(b'unexpected value in setenv request')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	581 self.ui.log(b'chgserver', b'setenv: %r\n', sorted(newenv.keys()))
43860 5e0f6451e2d2 chg: fix chg to work with py3.7+ "coercing" the locale Kyle Lippincott <spectral@google.com> parents: 43621 diff changeset	582
30635 a150173da1c1 py3: replace os.environ with encoding.environ (part 2 of 5) Pulkit Goyal <7895pulkit@gmail.com> parents: 30619 diff changeset	583 encoding.environ.clear()
a150173da1c1 py3: replace os.environ with encoding.environ (part 2 of 5) Pulkit Goyal <7895pulkit@gmail.com> parents: 30619 diff changeset	584 encoding.environ.update(newenv)
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	585
980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	586 capabilities = commandserver.server.capabilities.copy()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	587 capabilities.update(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	588 {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	589 b'attachio': attachio,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	590 b'chdir': chdir,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	591 b'runcommand': runcommand,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	592 b'setenv': setenv,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	593 b'setumask': setumask,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	594 b'setumask2': setumask2,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	595 }
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	596 )
27792 980957333cfa chgserver: import background server extension from cHg Yuya Nishihara <yuya@tcha.org> parents: diff changeset	597
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	598 if util.safehasattr(procutil, b'setprocname'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	599
30750 378686afca52 chgserver: add the setprocname interface Jun Wu <quark@fb.com> parents: 30741 diff changeset	600 def setprocname(self):
378686afca52 chgserver: add the setprocname interface Jun Wu <quark@fb.com> parents: 30741 diff changeset	601 """Change process title"""
378686afca52 chgserver: add the setprocname interface Jun Wu <quark@fb.com> parents: 30741 diff changeset	602 name = self._readstr()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	603 self.ui.log(b'chgserver', b'setprocname: %r\n', name)
37120 a8a902d7176e procutil: bulk-replace function calls to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37119 diff changeset	604 procutil.setprocname(name)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	605
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	606 capabilities[b'setprocname'] = setprocname
30750 378686afca52 chgserver: add the setprocname interface Jun Wu <quark@fb.com> parents: 30741 diff changeset	607
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	608
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	609 def _tempaddress(address):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	610 return b'%s.%d.tmp' % (address, os.getpid())
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	611
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	612
28326 ea400a4f32e6 chgserver: mangle server address to include confighash Jun Wu <quark@fb.com> parents: 28325 diff changeset	613 def _hashaddress(address, hashstr):
30619 88efb4fb1975 chgserver: truncate base address at "." for hash address Jun Wu <quark@fb.com> parents: 30618 diff changeset	614 # if the basename of address contains '.', use only the left part. this
88efb4fb1975 chgserver: truncate base address at "." for hash address Jun Wu <quark@fb.com> parents: 30618 diff changeset	615 # makes it possible for the client to pass 'server.tmp$PID' and follow by
88efb4fb1975 chgserver: truncate base address at "." for hash address Jun Wu <quark@fb.com> parents: 30618 diff changeset	616 # an atomic rename to avoid locking when spawning new servers.
88efb4fb1975 chgserver: truncate base address at "." for hash address Jun Wu <quark@fb.com> parents: 30618 diff changeset	617 dirname, basename = os.path.split(address)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	618 basename = basename.split(b'.', 1)[0]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	619 return b'%s-%s' % (os.path.join(dirname, basename), hashstr)
28326 ea400a4f32e6 chgserver: mangle server address to include confighash Jun Wu <quark@fb.com> parents: 28325 diff changeset	620
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	621
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	622 class chgunixservicehandler(object):
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	623 """Set of operations for chg services"""
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	624
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	625 pollinterval = 1 # [sec]
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	626
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	627 def __init__(self, ui):
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	628 self.ui = ui
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	629 self._idletimeout = ui.configint(b'chgserver', b'idletimeout')
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	630 self._lastactive = time.time()
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	631
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	632 def bindsocket(self, sock, address):
29597 581c0c7cb258 chgserver: refactor initialization of real/base addresses Yuya Nishihara <yuya@tcha.org> parents: 29596 diff changeset	633 self._inithashstate(address)
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	634 self._checkextensions()
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	635 self._bind(sock)
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	636 self._createsymlink()
32237 1ada3d18e7fb commandserver: move printbanner logic to bindsocket Jun Wu <quark@fb.com> parents: 32236 diff changeset	637 # no "listening at" message should be printed to simulate hg behavior
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	638
29597 581c0c7cb258 chgserver: refactor initialization of real/base addresses Yuya Nishihara <yuya@tcha.org> parents: 29596 diff changeset	639 def _inithashstate(self, address):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	640 self._baseaddress = address
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	641 if self.ui.configbool(b'chgserver', b'skiphash'):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	642 self._hashstate = None
a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	643 self._realaddress = address
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	644 return
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	645 self._hashstate = hashstate.fromui(self.ui)
a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	646 self._realaddress = _hashaddress(address, self._hashstate.confighash)
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	647
28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	648 def _checkextensions(self):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	649 if not self._hashstate:
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	650 return
28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	651 if extensions.notloaded():
28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	652 # one or more extensions failed to load. mtimehash becomes
28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	653 # meaningless because we do not know the paths of those extensions.
28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	654 # set mtimehash to an illegal hash value to invalidate the server.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	655 self._hashstate.mtimehash = b''
29545 28aca3fafc2a chgserver: reorder service classes to make future patches readable Yuya Nishihara <yuya@tcha.org> parents: 29543 diff changeset	656
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	657 def _bind(self, sock):
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	658 # use a unique temp address so we can stat the file and do ownership
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	659 # check later
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	660 tempaddress = _tempaddress(self._realaddress)
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	661 util.bindunixsocket(sock, tempaddress)
29529 02de1dbd4f6e chgserver: narrow scope of chdir() to socket.bind() Yuya Nishihara <yuya@tcha.org> parents: 29513 diff changeset	662 self._socketstat = os.stat(tempaddress)
32236 c8b9943c07eb commandserver: move "listen" responsibility from service to handler Jun Wu <quark@fb.com> parents: 32208 diff changeset	663 sock.listen(socket.SOMAXCONN)
29529 02de1dbd4f6e chgserver: narrow scope of chdir() to socket.bind() Yuya Nishihara <yuya@tcha.org> parents: 29513 diff changeset	664 # rename will replace the old socket file if exists atomically. the
02de1dbd4f6e chgserver: narrow scope of chdir() to socket.bind() Yuya Nishihara <yuya@tcha.org> parents: 29513 diff changeset	665 # old server will detect ownership change and exit.
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	666 util.rename(tempaddress, self._realaddress)
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	667
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	668 def _createsymlink(self):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	669 if self._baseaddress == self._realaddress:
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	670 return
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	671 tempaddress = _tempaddress(self._baseaddress)
a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	672 os.symlink(os.path.basename(self._realaddress), tempaddress)
a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	673 util.rename(tempaddress, self._baseaddress)
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	674
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	675 def _issocketowner(self):
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	676 try:
36781 ffa3026d4196 cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime Augie Fackler <augie@google.com> parents: 35582 diff changeset	677 st = os.stat(self._realaddress)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	678 return (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	679 st.st_ino == self._socketstat.st_ino
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	680 and st[stat.ST_MTIME] == self._socketstat[stat.ST_MTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	681 )
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	682 except OSError:
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	683 return False
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	684
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	685 def unlinksocket(self, address):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	686 if not self._issocketowner():
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	687 return
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	688 # it is possible to have a race condition here that we may
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	689 # remove another server's socket file. but that's okay
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	690 # since that server will detect and exit automatically and
0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	691 # the client will start a new server on demand.
31545 82b3ec04b652 chgserver: use tryunlink Ryan McElroy <rmcelroy@fb.com> parents: 31108 diff changeset	692 util.tryunlink(self._realaddress)
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	693
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	694 def shouldexit(self):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	695 if not self._issocketowner():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	696 self.ui.log(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	697 b'chgserver', b'%s is not owned, exiting.\n', self._realaddress
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	698 )
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	699 return True
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	700 if time.time() - self._lastactive > self._idletimeout:
40828 25e9089c7686 commandserver: turn server debug messages into logs Yuya Nishihara <yuya@tcha.org> parents: 40825 diff changeset	701 self.ui.log(b'chgserver', b'being idle too long. exiting.\n')
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	702 return True
71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	703 return False
71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	704
71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	705 def newconnection(self):
29598 a67398726747 chgserver: rename private functions and variables of chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29597 diff changeset	706 self._lastactive = time.time()
29596 71c197d82b7e chgserver: reorder functions in chgunixservicehandler Yuya Nishihara <yuya@tcha.org> parents: 29595 diff changeset	707
40875 e7110f44ee2d commandserver: pass around option to hook repo instance creation Yuya Nishihara <yuya@tcha.org> parents: 40828 diff changeset	708 def createcmdserver(self, repo, conn, fin, fout, prereposetups):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	709 return chgcmdserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	710 self.ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	711 repo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	712 fin,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	713 fout,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	714 conn,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	715 prereposetups,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	716 self._hashstate,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	717 self._baseaddress,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	718 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42818 diff changeset	719
28223 0a853dc9b306 chgserver: auto exit after being idle for too long or lose the socket file Jun Wu <quark@fb.com> parents: 28159 diff changeset	720
29546 a7513390a9db chgserver: extract stub factory of service object Yuya Nishihara <yuya@tcha.org> parents: 29545 diff changeset	721 def chgunixservice(ui, repo, opts):
33860 3cfc9070245f demandimport: disable if chg is being used Jun Wu <quark@fb.com> parents: 33499 diff changeset	722 # CHGINTERNALMARK is set by chg client. It is an indication of things are
3cfc9070245f demandimport: disable if chg is being used Jun Wu <quark@fb.com> parents: 33499 diff changeset	723 # started by chg so other code can do things accordingly, like disabling
3cfc9070245f demandimport: disable if chg is being used Jun Wu <quark@fb.com> parents: 33499 diff changeset	724 # demandimport or detecting chg client started by chg client. When executed
3cfc9070245f demandimport: disable if chg is being used Jun Wu <quark@fb.com> parents: 33499 diff changeset	725 # here, CHGINTERNALMARK is no longer useful and hence dropped to make
3cfc9070245f demandimport: disable if chg is being used Jun Wu <quark@fb.com> parents: 33499 diff changeset	726 # environ cleaner.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	727 if b'CHGINTERNALMARK' in encoding.environ:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	728 del encoding.environ[b'CHGINTERNALMARK']
44261 04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	729 # Python3.7+ "coerces" the LC_CTYPE environment variable to a UTF-8 one if
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	730 # it thinks the current value is "C". This breaks the hash computation and
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	731 # causes chg to restart loop.
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	732 if b'CHGORIG_LC_CTYPE' in encoding.environ:
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	733 encoding.environ[b'LC_CTYPE'] = encoding.environ[b'CHGORIG_LC_CTYPE']
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	734 del encoding.environ[b'CHGORIG_LC_CTYPE']
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	735 elif b'CHG_CLEAR_LC_CTYPE' in encoding.environ:
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	736 if b'LC_CTYPE' in encoding.environ:
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	737 del encoding.environ[b'LC_CTYPE']
04a3ae7aba14 chg: force-set LC_CTYPE on server start to actual value from the environment Kyle Lippincott <spectral@google.com> parents: 44060 diff changeset	738 del encoding.environ[b'CHG_CLEAR_LC_CTYPE']
30511 1873563e1ede chgserver: drop CHGINTERNALMARK by chgunixservice() Yuya Nishihara <yuya@tcha.org> parents: 30507 diff changeset	739
29546 a7513390a9db chgserver: extract stub factory of service object Yuya Nishihara <yuya@tcha.org> parents: 29545 diff changeset	740 if repo:
30332 318a24b52eeb spelling: fixes of non-dictionary words Mads Kiilerich <madski@unity3d.com> parents: 29841 diff changeset	741 # one chgserver can serve multiple repos. drop repo information
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	742 ui.setconfig(b'bundle', b'mainreporoot', b'', b'repo')
29547 300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	743 h = chgunixservicehandler(ui)
300318b7d66d chgserver: switch to new forking service Yuya Nishihara <yuya@tcha.org> parents: 29546 diff changeset	744 return commandserver.unixforkingservice(ui, repo=None, opts=opts, handler=h)

Mercurial > public > mercurial-scm > hg

annotate mercurial/chgserver.py @ 45892:06b64fabf91c