# Can't upgrade py25-cloud-init, "No module named chardet"



## sremick (Aug 23, 2019)

So I'm trying to update ports on a particular system and it's dying on py25-cloud-init:

```
running build_scripts
creating build/scripts-2.7
copying tools/cloud-init-per -> build/scripts-2.7
(cd /usr/ports/net/cloud-init/work-py27/cloud-init-19.2 && /bin/mv sysvinit/freebsd/cloudconfig config/cloud.cfg)
===>>> Building the port required 2 seconds
===>  Staging for py27-cloud-init-19.2
===>   Generating temporary packing list
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 146, in <module>
    if (is_f(f) and not is_generator(f))],
  File "setup.py", line 123, in render_tmpl
    tiny_p([sys.executable, './tools/render-cloudcfg', template, fpath])
  File "setup.py", line 50, in tiny_p
    (cmd, ret, out, err))
RuntimeError: Failed running ['/usr/local/bin/python2.7', './tools/render-cloudcfg', 'systemd/cloud-init-local.service.tmpl', 'RENDERED_TEMPDLRpVL8/cloud-init-local.service'] [rc=1] (, Traceback (most recent call last):
  File "./tools/render-cloudcfg", line 12, in <module>
    from cloudinit import templater
  File "/usr/ports/net/cloud-init/work-py27/cloud-init-19.2/cloudinit/templater.py", line 36, in <module>
    from cloudinit import util
  File "/usr/ports/net/cloud-init/work-py27/cloud-init-19.2/cloudinit/util.py", line 49, in <module>
    from cloudinit import url_helper
  File "/usr/ports/net/cloud-init/work-py27/cloud-init-19.2/cloudinit/url_helper.py", line 13, in <module>
    import requests
  File "/usr/local/lib/python2.7/site-packages/requests/__init__.py", line 44, in <module>
    import chardet
ImportError: No module named chardet
)
*** Error code 1

Stop.
make: stopped in /usr/ports/net/cloud-init

===>>> make stage failed for net/cloud-init@py27
===>>> Aborting update


===>>> You can restart from the point of failure with this command line:
       portmaster <flags> net/cloud-init@py27

This command has been saved to /tmp/portmasterfail.txt
```
Google searching hasn't come up with anything helpful, and emailing the port maintainer a few weeks ago hasn't gotten a response. Any ideas? FreeBSD 12-RELEASE


----------



## T-Daemon (Aug 23, 2019)

Install textproc/py-chardet (two flavors: py36-chardet, py27-chardet).

Or, net/cloud-init also comes in two flavors. If you don't need py27-cloud-init you can exclude it from build.


----------



## sremick (Aug 24, 2019)

T-Daemon said:


> Install textproc/py-chardet (two flavors: py36-chardet, py27-chardet).


It's actually already installed:

```
py37-chardet-3.0.4_1           Universal encoding detector for Python 2 and 3
```



> Or, net/cloud-init also comes in two flavors. If you don't need py27-cloud-init you can exclude it from build.


I need to dig back to see why it got installed. pkg info -r doesn't list anything depending on it but something must have said it needed to be installed at some point.


----------



## T-Daemon (Aug 24, 2019)

sremick said:


> It's actually already installed:





sremick said:


> py37-chardet-3.0.4_1



The build processs is terminated trying to import chardet to build py27-cloud-init. Is py27-chardet-3.0.4_1 installed as well?


----------



## sremick (Sep 10, 2019)

T-Daemon said:


> The build processs is terminated trying to import chardet to build py27-cloud-init. Is py27-chardet-3.0.4_1 installed as well?


Sorry, life stuff came up and I had to shelve this for a bit. Back to pick up where we left off.

To answer your question: no, the py37 version is:


```
# pkg version | grep chardet
py37-chardet-3.0.4_1               ?
```


----------



## T-Daemon (Sep 10, 2019)

Then isn't it the logical conclusion to install py27-chardet as well to satisfy the build process?


----------



## sremick (Sep 10, 2019)

T-Daemon said:


> Then isn't it the logical conclusion to install py27-chardet as well to satisfy the build process?


Thanks. Sorry to seem dense. This did open my eyes to the whole Python "flavors" situation, which I was not familiar with before. I was able to force a build of the py27 flavor of chardet, which then allowed the portmaster updating process to proceed further, but then it barfed again elsewhere for what is probably a related issue.

I think the crux of my problem is the whole py37/py27 mix. Perhaps initially everything installed py27, but then some things upgraded to py37 and now there's a mix and things aren't happy about it. Here is the list of my py37 ports:

```
py37-Babel-2.7.0                   ?
py37-Jinja2-2.10.1                 ?
py37-MarkupSafe-1.1.1              ?
py37-alabaster-0.7.6               ?
py37-asn1crypto-0.24.0             ?
py37-attrs-19.1.0                  ?
py37-blinker-1.4                   ?
py37-boto-2.49.0_1                 ?
py37-certifi-2019.6.16             ?
py37-cffi-1.12.3                   ?
py37-chardet-3.0.4_1               ?
py37-configobj-5.0.6_1             ?
py37-cryptography-2.6.1            ?
py37-cython-0.29_1                 =
py37-docutils-0.15.2               ?
py37-idna-2.8                      ?
py37-imagesize-1.1.0               ?
py37-jsonpatch-1.21_1              ?
py37-jsonpointer-1.9_2             ?
py37-jsonschema-3.0.2              ?
py37-oauthlib-1.1.2                ?
py37-openssl-19.0.0                ?
py37-pycparser-2.19                ?
py37-pygments-2.4.1                ?
py37-pyjwt-1.4.0_1                 ?
py37-pyrsistent-0.14.11            ?
py37-pysocks-1.7.0                 ?
py37-pystemmer-1.3.0_2             ?
py37-pytest-runner-2.11.1          ?
py37-pytz-2019.2,1                 ?
py37-requests-2.21.0               ?
py37-serial-3.4_1                  ?
py37-setuptools-41.2.0             =
py37-setuptools_scm-3.3.3          ?
py37-six-1.12.0                    ?
py37-snowballstemmer-1.2.1         ?
py37-sphinx-1.6.5_2,1              ?
py37-sphinx_rtd_theme-0.4.3        ?
py37-sphinxcontrib-websupport-1.1.2 ?
py37-urllib3-1.22,1                ?
py37-vcversioner-2.16.0.0_1        ?
py37-yaml-5.1                      ?
```

And then my py27 ports:

```
py27-Automat-0.7.0_1               =
py27-Babel-2.7.0                   =
py27-Jinja2-2.10.1                 =
py27-MarkupSafe-1.1.1              =
py27-Paste-2.0.3                   =
py27-PyHamcrest-1.9.0_1            =
py27-acme-0.38.0,1                 =
py27-asn1crypto-0.24.0             =
py27-attrs-19.1.0                  =
py27-bcrypt-3.1.7                  =
py27-blinker-1.4                   =
py27-boto-2.49.0_1                 =
py27-canonicaljson-1.1.4           =
py27-certbot-0.38.0,1              =
py27-certifi-2019.6.16             =
py27-cffi-1.12.3                   =
py27-characteristic-14.3.0         =
py27-chardet-3.0.4_1               =
py27-cheetah-2.4.4_1               =
py27-cloud-init-19.2               =
py27-configargparse-0.14.0         =
py27-configobj-5.0.6_1             =
py27-constantly-15.1.0             =
py27-cryptography-2.6.1            =
py27-daemonize-2.5.0               =
py27-dateutil-2.8.0                =
py27-defusedxml-0.5.0              =
py27-distro-1.4.0_1                =
py27-docutils-0.15.2               =
py27-enum34-1.1.6                  =
py27-frozendict-1.2                =
py27-functools32-3.2.3_1           =
py27-future-0.17.1_1               =
py27-hyperlink-19.0.0              =
py27-idna-2.8                      =
py27-incremental-17.5.0            =
py27-ipaddress-1.0.22              =
py27-josepy-1.2.0                  =
py27-jsonpatch-1.21_1              =
py27-jsonpointer-1.9_2             =
py27-jsonschema-3.0.2              =
py27-m2r-0.2.1_1                   =
py27-markdown-2.6.11_1             =
py27-matrix-synapse-0.34.1.1       ?
py27-mistune-0.8.4                 =
py27-msgpack-0.6.1                 =
py27-netaddr-0.7.19_1              =
py27-oauth-1.0.1_2                 =
py27-oauthlib-1.1.2                =
py27-olefile-0.46                  =
py27-openssl-19.0.0                =
py27-parsedatetime-2.4_1           =
py27-phonenumbers-8.4.1            =
py27-pillow-6.0.0_2                =
py27-prettytable-0.7.2_2           =
py27-prometheus-client-0.7.1       =
py27-psutil-5.6.3                  =
py27-pyasn1-0.4.5                  =
py27-pyasn1-modules-0.2.5          =
py27-pycparser-2.19                =
py27-pyjwt-1.4.0_1                 =
py27-pymacaroons-pynacl-0.13.0     =
py27-pynacl-1.3.0                  =
py27-pyrfc3339-1.1                 =
py27-pyrsistent-0.14.11            =
py27-pysaml2-4.8.0                 =
py27-pysocks-1.7.0                 =
py27-pytest-runner-2.11.1          =
py27-pytz-2019.2,1                 =
py27-repoze.who-2.3                =
py27-requests-2.21.0               =
py27-requests-toolbelt-0.8.0       =
py27-serial-3.4_1                  =
py27-service_identity-18.1.0       =
py27-setuptools-41.2.0             =
py27-setuptools_scm-3.3.3          =
py27-signedjson-1.0.0_1            =
py27-simplejson-3.16.1             =
py27-six-1.12.0                    =
py27-sortedcontainers-2.1.0        =
py27-sqlite3-2.7.16_7              =
py27-tkinter-2.7.16_6              =
py27-treq-17.8.0                   =
py27-twisted-18.9.0_1              =
py27-unpaddedbase64-1.1.0          =
py27-urllib3-1.22,1                =
py27-vcversioner-2.16.0.0_1        =
py27-webob-1.8.2                   =
py27-yaml-5.1                      =
py27-zope.component-4.2.2          =
py27-zope.event-4.1.0              =
py27-zope.interface-4.6.0          =
```

You'll see that "matrix" is in there as py27-matrix. But now when portmaster tries to update it to py27-matrix-synapse-1.2.1, that's where things die now:

```
===>>> All >> net-im/py-matrix-synapse@py27 (1/6)

===>>> Currently installed version: py27-matrix-synapse-0.34.1.1
===>>> Port directory: /usr/ports/net-im/py-matrix-synapse@py27

===>>> Starting check for build dependencies
===>>> Gathering dependency list for net-im/py-matrix-synapse@py27 from ports
===>>> Dependency check complete for net-im/py-matrix-synapse@py27

===>>> All >> py27-matrix-synapse-0.34.1.1 (1/6)

===>  Cleaning for py27-matrix-synapse-1.2.1
===>  py27-matrix-synapse-1.2.1 Unknown flavor 'py27', possible flavors:
py37..
*** Error code 1

Stop.
make: stopped in /usr/ports/net-im/py-matrix-synapse

===>>> make build failed for net-im/py-matrix-synapse@py27
===>>> Aborting update

===>>> Update for net-im/py-matrix-synapse@py27 failed
===>>> Aborting update
```

Unfortunately I'm not familiar enough with the python "flavor" situation to know the proper way to untangle this.  This is a single-task server that I'm setting up for the first time and has never actually worked yet, so aggressive nuking/resetting is entirely permissible.


----------



## SirDice (Sep 11, 2019)

sremick said:


> This is a single-task server that I'm setting up for the first time and has never actually worked yet, so aggressive nuking/resetting is entirely permissible.


I would suggest switching to packages and just run `pkg upgrade -f` to upgrade/re-install the whole lot.


----------



## sremick (Sep 13, 2019)

SirDice said:


> I would suggest switching to packages and just run `pkg upgrade -f` to upgrade/re-install the whole lot.


Well, I tried that but it rolled me back to an earlier issue. In packages, the latest version of Synapse is 0.34.1.1. I was having trouble setting that one up, so when I chatted w/ the Matrix folks in IRC they pointed out that that version is ancient and that I should get current. In ports, there is Synapse 1.2.1 which is exponentially newer. However this has dependencies on the py36- versions of everything. So it seems I'm stuck going with the ports versions of everything and somehow untangling it so it actually works.


----------



## SirDice (Sep 16, 2019)

You can switch to the latest package repository. By default you're on quarterly. 

Create a /usr/local/etc/pkg/repos/FreeBSD.conf:

```
FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest"
}
```

The latest package repositories follow the ports tree more closely, the quarterly repositories are update once every three months.


----------

