Bug 13837 - Samba 4.9.5 - Error on samba-tool with python 2.6
Summary: Samba 4.9.5 - Error on samba-tool with python 2.6
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: Python (show other bugs)
Version: 4.9.5
Hardware: x64 Linux
: P5 normal (vote)
Target Milestone: ---
Assignee: Karolin Seeger
QA Contact: Samba QA Contact
URL:
Keywords:
: 13845 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-03-14 00:12 UTC by Miguel Medalha
Modified: 2019-04-04 07:25 UTC (History)
3 users (show)

See Also:


Attachments
patch graph.py to work with ALL pythons (921 bytes, patch)
2019-03-19 23:05 UTC, Douglas Bagnall
no flags Details
Patch for bug 13845 variant (834 bytes, patch)
2019-03-19 23:19 UTC, Douglas Bagnall
no flags Details
Patch for kcc_utils (1.17 KB, patch)
2019-03-22 02:27 UTC, Douglas Bagnall
no flags Details
All three patches, with the master commit hash where applicable (3.20 KB, patch)
2019-03-22 23:01 UTC, Douglas Bagnall
garming: review+
Details
Additional patch for 2.6 compatible logging (980 bytes, patch)
2019-03-26 20:52 UTC, Douglas Bagnall
no flags Details
All four patches with master commit IDs where applicable (4.21 KB, patch)
2019-03-26 22:11 UTC, Douglas Bagnall
garming: review+
Details
The two patches needed for 4.9 (2.38 KB, patch)
2019-03-27 04:05 UTC, Douglas Bagnall
abartlet: review+
Details
4.9 patch with reviewed-by tags (2.43 KB, patch)
2019-03-27 04:13 UTC, Douglas Bagnall
dbagnall: review? (abartlet)
Details
Patches for 4.10, with reviewed-by tags (4.36 KB, patch)
2019-03-27 04:17 UTC, Douglas Bagnall
garming: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Miguel Medalha 2019-03-14 00:12:37 UTC
Samba Release 4.9.5

"samba-tool" (without parameters), "samba-tool --help", and "samba-tool visualize" produce the following error:


ERROR(<type 'exceptions.SyntaxError'>): uncaught exception - invalid syntax (graph.py, line 91)
  File "/usr/local/samba/bin/samba-tool", line 45, in <module>
    retval = cmd._run("samba-tool", subcommand, *args)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 247, in _run
    cmd = self.subcommands[cmd_name]
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/main.py", line 35, in __getitem__
    fromlist=['cmd_%s' % attr]),
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/visualize.py", line 33, in <module>
    from samba.graph import dot_graph


The other sub-commands of "samba-tool" don't give the error.

OS is CentOS 6.10, which uses Python 2.x.
Comment 1 Andrew Bartlett 2019-03-19 22:17:56 UTC
*** Bug 13845 has been marked as a duplicate of this bug. ***
Comment 2 Douglas Bagnall 2019-03-19 23:05:09 UTC
Created attachment 14952 [details]
patch graph.py to work with ALL pythons

Could you try this patch?
Comment 3 Douglas Bagnall 2019-03-19 23:19:19 UTC
Created attachment 14953 [details]
Patch for bug 13845 variant

The second patch is for the erstwhile bug 13845.
Comment 4 Miguel Medalha 2019-03-20 12:59:36 UTC
Samba(In reply to Douglas Bagnall from comment #2)

Tested Samba 4.9.5 with the patch.
With "samba-tool" the error is now the following:

ERROR(<type 'exceptions.ImportError'>): uncaught exception - cannot import name Counter
  File "/usr/local/samba/bin/samba-tool", line 45, in <module>
    retval = cmd._run("samba-tool", subcommand, *args)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 247, in _run
    cmd = self.subcommands[cmd_name]
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/main.py", line 35, in __getitem__
    fromlist=['cmd_%s' % attr]),
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/visualize.py", line 39, in <module>
    from samba.kcc import KCC, ldif_import_export
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/__init__.py", line 32, in <module>
    from samba.kcc.kcc_utils import Site, Partition, Transport, SiteLink
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/kcc_utils.py", line 34, in <module>
    from collections import Counter


With "samba-tool visuzlize" the error is the following:

ERROR(<type 'exceptions.ImportError'>): uncaught exception - cannot import name Counter
  File "/usr/local/samba/bin/samba-tool", line 45, in <module>
    retval = cmd._run("samba-tool", subcommand, *args)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 226, in _run
    return self.subcommands[subcommand]._run(
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/main.py", line 35, in __getitem__
    fromlist=['cmd_%s' % attr]),
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/visualize.py", line 39, in <module>
    from samba.kcc import KCC, ldif_import_export
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/__init__.py", line 32, in <module>
    from samba.kcc.kcc_utils import Site, Partition, Transport, SiteLink
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/kcc_utils.py", line 34, in <module>
    from collections import Counter
Comment 5 Miguel Medalha 2019-03-20 13:13:59 UTC
Tested Samba 4.10.0 with the patches.

With "samba-tool" the error is now the following:

ERROR(<type 'exceptions.ImportError'>): uncaught exception - cannot import name Counter
  File "/usr/local/samba/bin/samba-tool", line 45, in <module>
    retval = cmd._run("samba-tool", subcommand, *args)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 254, in _run
    cmd = self.subcommands[cmd_name]
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/main.py", line 36, in __getitem__
    fromlist=['cmd_%s' % attr]),
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/drs.py", line 42, in <module>
    from samba.uptodateness import (
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/uptodateness.py", line 29, in <module>
    from samba.kcc import KCC
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/__init__.py", line 32, in <module>
    from samba.kcc.kcc_utils import Site, Partition, Transport, SiteLink
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/kcc_utils.py", line 34, in <module>
    from collections import Counter


With "samba-tool visuzlize" the error is the following:

ERROR(<type 'exceptions.ImportError'>): uncaught exception - cannot import name Counter
  File "/usr/local/samba/bin/samba-tool", line 45, in <module>
    retval = cmd._run("samba-tool", subcommand, *args)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 233, in _run
    return self.subcommands[subcommand]._run(
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/main.py", line 36, in __getitem__
    fromlist=['cmd_%s' % attr]),
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/visualize.py", line 39, in <module>
    from samba.kcc import KCC, ldif_import_export
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/__init__.py", line 32, in <module>
    from samba.kcc.kcc_utils import Site, Partition, Transport, SiteLink
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/kcc/kcc_utils.py", line 34, in <module>
    from collections import Counter
Comment 6 Miguel Medalha 2019-03-20 13:17:00 UTC
I should add that Samba 4.10.0 was configured and built with:

PYTHON=python2 ./configure
PYTHON=python2 make
Comment 7 Douglas Bagnall 2019-03-22 02:27:05 UTC
Created attachment 14976 [details]
Patch for kcc_utils

Does this additional patch help?
Comment 8 Miguel Medalha 2019-03-22 10:10:46 UTC
(In reply to Douglas Bagnall from comment #7)

Yes it does! Looks like everything is now working as it should.

I appreciate your good will on solving this.
Comment 9 Andrew Bartlett 2019-03-22 19:25:30 UTC
Please don't close bugs until we land the patches in the branches and make a release.

Thanks!
Comment 10 Miguel Medalha 2019-03-22 19:49:23 UTC
(In reply to Andrew Bartlett from comment #9)

Does status WORKSFORME have the same effect as RESOLVED? I thought that marking it as WORKSFORME would be just that, an information...
Comment 11 Andrew Bartlett 2019-03-22 20:13:57 UTC
(In reply to Miguel Medalha from comment #10)
WORKSFORME is a way for a Samba developer to indicate that a bug can't be reproduced.  

Therefore it is an end state without a change to the codebase, much like INVALID. 

I hope this clarifies things.
Comment 12 Douglas Bagnall 2019-03-22 23:01:46 UTC
Created attachment 14983 [details]
All three patches, with the master commit hash where applicable

This collects the three patches together with one of them pulled form master with the commit hash.


passed CI here: 

https://gitlab.com/samba-team/devel/samba/pipelines/53063206

but note that that test is not using Python 2.6. Our tester for 2.6 in this case is Miguel Medalha (thank you!).
Comment 13 Miguel Medalha 2019-03-25 19:29:04 UTC
I just discovered another issue with Samba 4.10.0 and Python 2.6 (CentOS 6.10).

Running "samba-tool ntacl sysvolcheck" produces the following:


ERROR(<type 'exceptions.TypeError'>): uncaught exception - __init__() got an unexpected keyword argument 'stream'
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 184, in _run
    return self.run(*args, **kwargs)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/ntacl.py", line 300, in run
    logger = self.get_logger()
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/netcmd/__init__.py", line 198, in get_logger
    **kwargs)
  File "/usr/local/samba/lib64/python2.6/site-packages/samba/logger.py", line 65, in get_samba_logger
    handler = logging.StreamHandler(stream=stream)


The same command produces no error with Python 2.7 under CentOS 7.6.
Comment 14 Douglas Bagnall 2019-03-26 20:52:18 UTC
Created attachment 14993 [details]
Additional patch for 2.6 compatible logging

This should fix that one.
Comment 15 Douglas Bagnall 2019-03-26 22:11:08 UTC
Created attachment 14994 [details]
All four patches with master commit IDs where applicable
Comment 16 Douglas Bagnall 2019-03-27 04:05:00 UTC
Created attachment 15002 [details]
The two patches needed for 4.9

I forgot this bug was initially about 4.9.
Two of the four 4.10 patches are also needed in 4.9.
Comment 17 Andrew Bartlett 2019-03-27 04:13:12 UTC
For clarity, the reason some of these do not go via master is that in master we only need to support Python3 so these changes are pointless and make the code worse.
Comment 18 Douglas Bagnall 2019-03-27 04:13:36 UTC
Created attachment 15003 [details]
4.9 patch with reviewed-by tags

This time the patch retains Garming's reviewed by
Comment 19 Douglas Bagnall 2019-03-27 04:17:39 UTC
Created attachment 15004 [details]
Patches for 4.10, with reviewed-by tags
Comment 20 Douglas Bagnall 2019-03-27 21:39:03 UTC
hi Karolin,

There are patchsets here for 4.9 and 4.10 that fix bugs affecting users with Python 2.6.
Comment 21 Karolin Seeger 2019-03-28 08:29:27 UTC
(In reply to Douglas Bagnall from comment #20)
Hi Douglas,

pushed to autobuild-v4-{10,9}-test.
Comment 22 Karolin Seeger 2019-04-02 07:56:41 UTC
(In reply to Karolin Seeger from comment #21)
Pushed to v4-9-test, pushed again to autobuild-v4-10-test.
Comment 23 Karolin Seeger 2019-04-03 10:29:25 UTC
(In reply to Karolin Seeger from comment #22)
Pushed to v4-10-test.
Closing out bug report.

Thanks!