[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH RFC-for-4.12] tools/xen-foreign: Update python scripts to be Py3 compatible


  • To: Wei Liu <wei.liu2@xxxxxxxxxx>, Juergen Gross <jgross@xxxxxxxx>
  • From: George Dunlap <george.dunlap@xxxxxxxxxx>
  • Date: Tue, 5 Mar 2019 12:19:25 +0000
  • Autocrypt: addr=george.dunlap@xxxxxxxxxx; prefer-encrypt=mutual; keydata= mQINBFPqG+MBEACwPYTQpHepyshcufo0dVmqxDo917iWPslB8lauFxVf4WZtGvQSsKStHJSj 92Qkxp4CH2DwudI8qpVbnWCXsZxodDWac9c3PordLwz5/XL41LevEoM3NWRm5TNgJ3ckPA+J K5OfSK04QtmwSHFP3G/SXDJpGs+oDJgASta2AOl9vPV+t3xG6xyfa2NMGn9wmEvvVMD44Z7R W3RhZPn/NEZ5gaJhIUMgTChGwwWDOX0YPY19vcy5fT4bTIxvoZsLOkLSGoZb/jHIzkAAznug Q7PPeZJ1kXpbW9EHHaUHiCD9C87dMyty0N3TmWfp0VvBCaw32yFtM9jUgB7UVneoZUMUKeHA fgIXhJ7I7JFmw3J0PjGLxCLHf2Q5JOD8jeEXpdxugqF7B/fWYYmyIgwKutiGZeoPhl9c/7RE Bf6f9Qv4AtQoJwtLw6+5pDXsTD5q/GwhPjt7ohF7aQZTMMHhZuS52/izKhDzIufl6uiqUBge 0lqG+/ViLKwCkxHDREuSUTtfjRc9/AoAt2V2HOfgKORSCjFC1eI0+8UMxlfdq2z1AAchinU0 eSkRpX2An3CPEjgGFmu2Je4a/R/Kd6nGU8AFaE8ta0oq5BSFDRYdcKchw4TSxetkG6iUtqOO ZFS7VAdF00eqFJNQpi6IUQryhnrOByw+zSobqlOPUO7XC5fjnwARAQABtCRHZW9yZ2UgVy4g RHVubGFwIDxkdW5sYXBnQHVtaWNoLmVkdT6JAlcEEwEKAEECGwMFCwkIBwMFFQoJCAsFFgID AQACHgECF4ACGQEWIQTXqBy2bTNXPzpOYFimNjwxBZC0bQUCXEowWQUJDCJ7dgAKCRCmNjwx BZC0beKvEACJ75YlJXd7TnNHgFyiCJkm/qPeoQ3sFGSDZuZh7SKcdt9+3V2bFEb0Mii1hQaz 3hRqZb8sYPHJrGP0ljK09k3wf8k3OuNxziLQBJyzvn7WNlE4wBEcy/Ejo9TVBdA4ph5D0YaZ nqdsPmxe/xlTFuSkgu4ep1v9dfVP1TQR0e+JIBa/Ss+cKC5intKm+8JxpOploAHuzaPu0L/X FapzsIXqgT9eIQeBEgO2hge6h9Jov3WeED/vh8kA7f8c6zQ/gs5E7VGALwsiLrhr0LZFcKcw kI3oCCrB/C/wyPZv789Ra8EXbeRSJmTjcnBwHRPjnjwQmetRDD1t+VyrkC6uujT5jmgOBzaj KCqZ8PcMAssOzdzQtKmjUQ2b3ICPs2X13xZ5M5/OVs1W3TG5gkvMh4YoHi4ilFnOk+v3/j7q 65FG6N0JLb94Ndi80HkIOQQ1XVGTyu6bUPaBg3rWK91Csp1682kD/dNVF3FKHrRLmSVtmEQR 5rK0+VGc/FmR6vd4haKGWIRuPxzg+pBR77avIZpU7C7+UXGuZ5CbHwIdY8LojJg2TuUdqaVj yxmEZLOA8rVHipCGrslRNthVbJrGN/pqtKjCClFZHIAYJQ9EGLHXLG9Pj76opfjHij3MpR3o pCGAh6KsCrfrsvjnpDwqSbngGyEVH030irSk4SwIqZ7FwLkBDQRUWmc6AQgAzpc8Ng5Opbrh iZrn69Xr3js28p+b4a+0BOvC48NfrNovZw4eFeKIzmI/t6EkJkSqBIxobWRpBkwGweENsqnd 0qigmsDw4N7J9Xx0h9ARDqiWxX4jr7u9xauI+CRJ1rBNO3VV30QdACwQ4LqhR/WA+IjdhyMH wj3EJGE61NdP/h0zfaLYAbvEg47/TPThFsm4m8Rd6bX7RkrrOgBbL/AOnYOMEivyfZZKX1vv iEemAvLfdk2lZt7Vm6X/fbKbV8tPUuZELzNedJvTTBS3/l1FVz9OUcLDeWhGEdlxqXH0sYWh E9+PXTAfz5JxKH+LMetwEM8DbuOoDIpmIGZKrZ+2fQARAQABiQNbBBgBCgAmAhsCFiEE16gc tm0zVz86TmBYpjY8MQWQtG0FAlxKMJ4FCQnQ/OQBKcBdIAQZAQoABgUCVFpnOgAKCRCyFcen x4Qb7cXrCAC0qQeEWmLa9oEAPa+5U6wvG1t/mi22gZN6uzQXH1faIOoDehr7PPESE6tuR/vI CTTnaSrd4UDPNeqOqVF07YexWD1LDcQG6PnRqC5DIX1RGE3BaSaMl2pFJP8y+chews11yP8G DBbxaIsTcHZI1iVIC9XLhoeegWi84vYc8F4ziADVfowbmbvcVw11gE8tmALCwTeBeZVteXjh 0OELHwrc1/4j4yvENjIXRO+QLIgk43kB57Upr4tP2MEcs0odgPM+Q+oETOJ00xzLgkTnLPim C1FIW2bOZdTj+Uq6ezRS2LKsNmW+PRRvNyA5ojEbA/faxmAjMZtLdSSSeFK8y4SoCRCmNjwx BZC0bevWEACRu+GyQgrdGmorUptniIeO1jQlpTiP5WpVnk9Oe8SiLoXUhXXNj6EtzyLGpYmf kEAbki+S6WAKnzZd3shL58AuMyDxtFNNjNeKJOcl6FL7JPBIIgIp3wR401Ep+/s5pl3Nw8Ii 157f0T7o8CPb54w6S1WsMkU78WzTxIs/1lLblSMcvyz1Jq64g4OqiWI85JfkzPLlloVf1rzy ebIBLrrmjhCE2tL1RONpE/KRVb+Q+PIs5+YcZ+Q1e0vXWA7NhTWFbWx3+N6WW6gaGpbFbopo FkYRpj+2TA5cX5zW148/xU5/ATEb5vdUkFLUFVy5YNUSyeBHuaf6fGmBrDc47rQjAOt1rmyD 56MUBHpLUbvA6NkPezb7T6bQpupyzGRkMUmSwHiLyQNJQhVe+9NiJJvtEE3jol0JVJoQ9WVn FAzPNCgHQyvbsIF3gYkCYKI0w8EhEoH5FHYLoKS6Jg880IY5rXzoAEfPvLXegy6mhYl+mNVN QUBD4h9XtOvcdzR559lZuC0Ksy7Xqw3BMolmKsRO3gWKhXSna3zKl4UuheyZtubVWoNWP/bn vbyiYnLwuiKDfNAinEWERC8nPKlv3PkZw5d3t46F1Dx0TMf16NmP+azsRpnMZyzpY8BL2eur feSGAOB9qjZNyzbo5nEKHldKWCKE7Ye0EPEjECS1gjKDwbkBDQRUWrq9AQgA7aJ0i1pQSmUR 6ZXZD2YEDxia2ByR0uZoTS7N0NYv1OjU8v6p017u0Fco5+Qoju/fZ97ScHhp5xGVAk5kxZBF DT4ovJd0nIeSr3bbWwfNzGx1waztfdzXt6n3MBKr7AhioB1m+vuk31redUdnhbtvN7O40MC+ fgSk5/+jRGxY3IOVPooQKzUO7M51GoOg4wl9ia3H2EzOoGhN2vpTbT8qCcL92ZZZwkBRldoA Wn7c1hEKSTuT3f1VpSmhjnX0J4uvKZ1V2R7rooKJYFBcySC0wa8aTmAtAvLgfcpe+legOtgq DKzLuN45xzEjyjCiI521t8zxNMPJY9FiCPNv0sCkDwARAQABiQI8BBgBCgAmAhsMFiEE16gc tm0zVz86TmBYpjY8MQWQtG0FAlxKNJYFCQnQrVkACgkQpjY8MQWQtG2Xxg//RrRP+PFYuNXt 9C5hec/JoY24TkGPPd2tMC9usWZVImIk7VlHlAeqHeE0lWU0LRGIvOBITbS9izw6fOVQBvCA Fni56S12fKLusWgWhgu03toT9ZGxZ9W22yfw5uThSHQ4y09wRWAIYvhJsKnPGGC2KDxFvtz5 4pYYNe8Icy4bwsxcgbaSFaRh+mYtts6wE9VzyJvyfTqbe8VrvE+3InG5rrlNn51AO6M4Wv20 iFEgYanJXfhicl0WCQrHyTLfdB5p1w+072CL8uryHQVfD0FcDe+J/wl3bmYze+aD1SlPzFoI MaSIXKejC6oh6DAT4rvU8kMAbX90T834Mvbc3jplaWorNJEwjAH/r+v877AI9Vsmptis+rni JwUissjRbcdlkKBisoUZRPmxQeUifxUpqgulZcYwbEC/a49+WvbaYUriaDLHzg9xisijHwD2 yWV8igBeg+cmwnk0mPz8tIVvwi4lICAgXob7HZiaqKnwaDXs4LiS4vdG5s/ElnE3rIc87yru 24n3ypeDZ6f5LkdqL1UNp5/0Aqbr3EiN7/ina4YVyscy9754l944kyHnnMRLVykg0v+kakj0 h0RJ5LbfLAMM8M52KIA3y14g0Fb7kHLcOUMVcgfQ3PrN6chtC+5l6ouDIlSLR3toxH8Aam7E rIFfe2Dk+lD9A9BVd2rfoHA=
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>, George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Tim Deegan <tim@xxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxx>, Julien Grall <julien.grall@xxxxxxx>, Jan Beulich <JBeulich@xxxxxxxx>, Ian Jackson <ian.jackson@xxxxxxxxxx>
  • Delivery-date: Tue, 05 Mar 2019 12:20:00 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 3/5/19 12:04 PM, Wei Liu wrote:
> On Tue, Mar 05, 2019 at 12:41:53PM +0100, Juergen Gross wrote:
>> On 04/03/2019 20:00, George Dunlap wrote:
>>> On 3/4/19 6:31 PM, Andrew Cooper wrote:
>>>> The issues are:
>>>>  * dict.has_key() was completely removed in Py3
>>>>  * dict.keys() is an iterable rather than list in Py3, so .sort() doesn't 
>>>> work.
>>>>  * list.sort(cmp=) was deprecated in Py2.4 and removed in Py3.  Replace it
>>>>    with a key= sort instead.
>>>>
>>>> This is all compatible with Py2.4 and later, which is when the sorted()
>>>> builtin was introduced.  Tested with Py2.7 and Py3.4
>>>
>>> Did you test ./configure with python3?
>>>
>>> Loads of the python-related runes in the configure are also
>>> python2-only, resulting in (at least for me) the wrong library paths
>>> being generated, so the following fails:
>>>
>>>     PYTHON=/usr/bin/python3 ./configure
>>>
>>> With the following message:
>>>
>>>   File "<string>", line 1
>>>     import distutils.sysconfig;     print
>>> distutils.sysconfig.get_config_var("VERSION")
>>>                                                   ^
>>> SyntaxError: invalid syntax
>>> checking for python3-config... /home/gdunlap/anaconda3/bin/python3-config
>>> checking Python.h usability... yes
>>> checking Python.h presence... yes
>>> checking for Python.h... yes
>>> checking for PyArg_ParseTuple in -lpython... no
>>> configure: error: Unable to find a suitable python development library
>>> configure: error: ./configure failed for tools
>>>
>>> Note the error with the VERSION above; that results in looking for
>>> `-lpython` rather than `-lpython3.7`, which results in the following
>>> line being executed:
>>>
>>> configure:7468: checking for PyArg_ParseTuple in -lpython
>>> configure:7493: gcc -o conftest -g -O2 -g -O2
>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>> -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell
>>> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe
>>> -fdebug-prefix-map==/usr/local/src/conda/-
>>> -fdebug-prefix-map==/usr/local/src/conda-prefix -fuse-linker-plugin
>>> -ffat-lto-objects -flto-partition=none -flto -flto -fuse-linker-plugin
>>> -ffat-lto-objects -flto-partition=none -g -DNDEBUG -fwrapv -O3 -Wall
>>> -L/persistent/local/home/dunlapg/anaconda3/lib/python3.7/config-3.7m-x86_64-linux-gnu
>>> -L/persistent/local/home/dunlapg/anaconda3/lib -lpython3.7m -lpthread
>>> -ldl  -lutil -lrt -lm  -Xlinker -export-dynamic conftest.c -lpython
>>> -llzma  -lpython3.7m -lpthread -ldl  -lutil -lrt -lm  -llzma  >&5
>>> /usr/bin/ld: cannot find -lpython
>>> collect2: error: ld returned 1 exit status
>>>
>>> The following fixes it for me:
>>>
>>> diff --git a/tools/configure b/tools/configure
>>> index acc857510e..d7e21db959 100755
>>> --- a/tools/configure
>>> +++ b/tools/configure
>>> @@ -6953,7 +6953,7 @@ if echo "$PYTHON" | grep -q "^/"; then :
>>>      PYTHON=`basename $PYTHONPATH`
>>>
>>>  elif test -z "$PYTHON"; then :
>>> -  PYTHON="python"
>>> +  PYTHON="python2"
>>>  else
>>>    as_fn_error $? "PYTHON specified, but is not an absolute path"
>>> "$LINENO" 5
>>>  fi
>>> diff --git a/tools/configure.ac b/tools/configure.ac
>>> index 1499344ce6..adf70aeb33 100644
>>> --- a/tools/configure.ac
>>> +++ b/tools/configure.ac
>>> @@ -355,7 +355,7 @@ esac
>>>  AS_IF([echo "$PYTHON" | grep -q "^/"], [
>>>      PYTHONPATH=$PYTHON
>>>      PYTHON=`basename $PYTHONPATH`
>>> -],[test -z "$PYTHON"], [PYTHON="python"],
>>> +],[test -z "$PYTHON"], [PYTHON="python2"],
>>>  [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
>>>  AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
>>>  AX_CHECK_PYTHON_VERSION([2], [3])
>>>
>>> ..but:
>>> 1) I'm not sure if we can guarantee that `python2` will exist on all
>>> systems we support,
>>> 2) We probably want to support python3 at some point anyway.
>>>
>>> On the other hand, Arch has python -> python3, but (apparently) has
>>> since 2011 [1]; so there's an argument to be made that this isn't
>>> something that needs to be solved for 4.12.
>>
>> Which raises the question whether Andrew's patch should be included
>> in 4.12. I'm tempted to refuse my Rab as I don't see why we need to
>> be python3 compliant in the build system while configure isn't.
>>
>> Thoughts?
> 
> I plan to have a closer look at python 3 compatibility issue today. But
> the outcome is unlikely to be simple -- most likely there will be a few
> patches. I don't think they will be suitable for 4.12.
> 
> We can document the python 2 requirement for 4.12.

Right, but there are two issues:
1. Build system requires python2
2. Build system assumes that 'python' == 'python2', and fails in strange
ways when it's not

I agree #1 isn't something to rush before the release.  Any opinions on #2?

Options are:

A. Simply document that if 'python'  isn't 'python2' (such as on Arch
Linux, or if you've got anaconda python installed), you need to
explicitly set PYTHON=/path/to/python2

B. Hard-code configure to look for python2 if not specified rather than
python.

C. Have configure detect when python is python3, and look for python2,
failing if it's not available.

Evaluation:

Option A isn't a great UI, but the situation has been like this for a
long time, so it's probably not the worst thing ever to wait for the
next release to fix things.  Also, serves you right for using such a
broken distro. :-)

Option B is dead-simple; I just don't know how far back it will be
compatible.

Option C is the most robust, but more work (though probably not a huge
amount).

Thoughts?

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.