StackStorm.st2smoketests pack reload fails due to non-ascii character #286
Description
Hello there,
when I run the ansible playbook reloading the services of st2 fails. This seems to be related to the content of the files (in this case /opt/stackstorm/packs/st2/aliases/actions_list.yaml) which contains a non-ascii character. Running st2ctl reload --register-all on the machine itself runs fine though. I've attached the stacktrace.
Ansible output
TASK [StackStorm.st2smoketests : Make sure packs are reloaded] *************************************************************************************************************************
fatal: [172.30.241.95]: FAILED! => {"changed": false, "cmd": ["st2ctl", "reload", "--register-all"], "delta": "0:00:05.209304", "end": "2021-01-14 10:29:29.030379", "msg": "non-zero return code", "rc": 1, "start": "2021-01-14 10:29:23.821075", "stderr": "2021-01-14 10:29:24,703 INFO [-] Connecting to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\".
2021-01-14 10:29:24,707 INFO [-] Successfully connected to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\".
2021-01-14 10:29:25,097 INFO [-] =========================================================
2021-01-14 10:29:25,097 INFO [-] ############## Registering triggers #####################
2021-01-14 10:29:25,097 INFO [-] =========================================================
2021-01-14 10:29:25,188 INFO [-] Registered 0 triggers.
2021-01-14 10:29:25,188 INFO [-] =========================================================
2021-01-14 10:29:25,188 INFO [-] ############## Registering sensors ######################
2021-01-14 10:29:25,188 INFO [-] =========================================================
2021-01-14 10:29:25,211 INFO [-] Registered 1 sensors.
2021-01-14 10:29:25,211 INFO [-] =========================================================
2021-01-14 10:29:25,211 INFO [-] ############## Registering runners ######################
2021-01-14 10:29:25,211 INFO [-] =========================================================
2021-01-14 10:29:26,180 INFO [-] Registered 14 runners.
2021-01-14 10:29:26,180 INFO [-] =========================================================
2021-01-14 10:29:26,180 INFO [-] ############## Registering actions ######################
2021-01-14 10:29:26,180 INFO [-] =========================================================
2021-01-14 10:29:28,205 INFO [-] Registered 81 actions.
2021-01-14 10:29:28,205 INFO [-] =========================================================
2021-01-14 10:29:28,205 INFO [-] ############## Registering rules ########################
2021-01-14 10:29:28,205 INFO [-] =========================================================
2021-01-14 10:29:28,234 INFO [-] Registered 1 rules.
2021-01-14 10:29:28,234 INFO [-] =========================================================
2021-01-14 10:29:28,234 INFO [-] ############## Registering aliases ######################
2021-01-14 10:29:28,234 INFO [-] =========================================================
2021-01-14 10:29:28,235 ERROR [-] Failed loading content from /opt/stackstorm/packs/st2/aliases/actions_list.yaml.
Traceback (most recent call last):
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load
return parser_func(fd)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load
return load(stream, SafeLoader)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load
loader = Loader(stream)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__
Reader.__init__(self, stream)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__
self.determine_encoding()
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding
self.update_raw()
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw
data = self.stream.read(size)
File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)
Traceback (most recent call last):
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 175, in _register_aliases_from_pack
self._register_action_alias(pack, alias)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 146, in _register_action_alias
action_alias=action_alias)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 117, in _get_action_alias_db
content = self._meta_loader.load(action_alias)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 236, in load
result = self._load(PARSER_FUNCS[file_ext], file_path)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load
return parser_func(fd)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load
return load(stream, SafeLoader)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load
loader = Loader(stream)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__
Reader.__init__(self, stream)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__
self.determine_encoding()
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding
self.update_raw()
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw
data = self.stream.read(size)
File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File \"/bin/st2-register-content\", line 22, in <module>
sys.exit(content_loader.main(sys.argv[1:]))
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 408, in main
register_content()
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 360, in register_content
register_aliases()
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 283, in register_aliases
raise e
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 279, in register_aliases
fail_on_failure=fail_on_failure
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 205, in register_aliases
result = registrar.register_from_packs(base_dirs=packs_base_paths)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 67, in register_from_packs
raise e
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 63, in register_from_packs
count = self._register_aliases_from_pack(pack=pack, aliases=aliases)
File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 180, in _register_aliases_from_pack
raise ValueError(msg)
ValueError: Failed to register alias \"/opt/stackstorm/packs/st2/aliases/actions_list.yaml\" from pack \"st2\": 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)", "stderr_lines": ["2021-01-14 10:29:24,703 INFO [-] Connecting to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\".", "2021-01-14 10:29:24,707 INFO [-] Successfully connected to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\".", "2021-01-14 10:29:25,097 INFO [-] =========================================================", "2021-01-14 10:29:25,097 INFO [-] ############## Registering triggers #####################", "2021-01-14 10:29:25,097 INFO [-] =========================================================", "2021-01-14 10:29:25,188 INFO [-] Registered 0 triggers.", "2021-01-14 10:29:25,188 INFO [-] =========================================================", "2021-01-14 10:29:25,188 INFO [-] ############## Registering sensors ######################", "2021-01-14 10:29:25,188 INFO [-] =========================================================", "2021-01-14 10:29:25,211 INFO [-] Registered 1 sensors.", "2021-01-14 10:29:25,211 INFO [-] =========================================================", "2021-01-14 10:29:25,211 INFO [-] ############## Registering runners ######################", "2021-01-14 10:29:25,211 INFO [-] =========================================================", "2021-01-14 10:29:26,180 INFO [-] Registered 14 runners.", "2021-01-14 10:29:26,180 INFO [-] =========================================================", "2021-01-14 10:29:26,180 INFO [-] ############## Registering actions ######################", "2021-01-14 10:29:26,180 INFO [-] =========================================================", "2021-01-14 10:29:28,205 INFO [-] Registered 81 actions.", "2021-01-14 10:29:28,205 INFO [-] =========================================================", "2021-01-14 10:29:28,205 INFO [-] ############## Registering rules ########################", "2021-01-14 10:29:28,205 INFO [-] =========================================================", "2021-01-14 10:29:28,234 INFO [-] Registered 1 rules.", "2021-01-14 10:29:28,234 INFO [-] =========================================================", "2021-01-14 10:29:28,234 INFO [-] ############## Registering aliases ######################", "2021-01-14 10:29:28,234 INFO [-] =========================================================", "2021-01-14 10:29:28,235 ERROR [-] Failed loading content from /opt/stackstorm/packs/st2/aliases/actions_list.yaml.", "Traceback (most recent call last):", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load", " return parser_func(fd)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load", " return load(stream, SafeLoader)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load", " loader = Loader(stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__", " Reader.__init__(self, stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__", " self.determine_encoding()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding", " self.update_raw()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw", " data = self.stream.read(size)", " File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode", " return codecs.ascii_decode(input, self.errors)[0]", "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)", "Traceback (most recent call last):", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 175, in _register_aliases_from_pack", " self._register_action_alias(pack, alias)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 146, in _register_action_alias", " action_alias=action_alias)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 117, in _get_action_alias_db", " content = self._meta_loader.load(action_alias)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 236, in load", " result = self._load(PARSER_FUNCS[file_ext], file_path)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load", " return parser_func(fd)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load", " return load(stream, SafeLoader)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load", " loader = Loader(stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__", " Reader.__init__(self, stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__", " self.determine_encoding()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding", " self.update_raw()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw", " data = self.stream.read(size)", " File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode", " return codecs.ascii_decode(input, self.errors)[0]", "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)", "", "During handling of the above exception, another exception occurred:", "", "Traceback (most recent call last):", " File \"/bin/st2-register-content\", line 22, in <module>", " sys.exit(content_loader.main(sys.argv[1:]))", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 408, in main", " register_content()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 360, in register_content", " register_aliases()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 283, in register_aliases", " raise e", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 279, in register_aliases", " fail_on_failure=fail_on_failure", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 205, in register_aliases", " result = registrar.register_from_packs(base_dirs=packs_base_paths)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 67, in register_from_packs", " raise e", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 63, in register_from_packs", " count = self._register_aliases_from_pack(pack=pack, aliases=aliases)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 180, in _register_aliases_from_pack", " raise ValueError(msg)", "ValueError: Failed to register alias \"/opt/stackstorm/packs/st2/aliases/actions_list.yaml\" from pack \"st2\": 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)"], "stdout": "Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]
##### st2 components status #####
st2actionrunner PID: 18661
st2actionrunner PID: 18663
st2actionrunner PID: 18665
st2actionrunner PID: 18667
st2actionrunner PID: 18669
st2actionrunner PID: 18671
st2actionrunner PID: 18673
st2actionrunner PID: 18675
st2actionrunner PID: 18677
st2actionrunner PID: 18679
st2api PID: 87707
st2api PID: 87772
st2stream PID: 87833
st2stream PID: 87852
st2auth PID: 19594
st2auth PID: 19665
st2garbagecollector PID: 18848
st2notifier PID: 18971
st2resultstracker is not running.
st2rulesengine PID: 19095
st2sensorcontainer PID: 19219
st2chatops is not running.
st2timersengine PID: 19964
st2workflowengine PID: 19843
st2scheduler PID: 19720", "stdout_lines": ["Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]", "##### st2 components status #####", "st2actionrunner PID: 18661", "st2actionrunner PID: 18663", "st2actionrunner PID: 18665", "st2actionrunner PID: 18667", "st2actionrunner PID: 18669", "st2actionrunner PID: 18671", "st2actionrunner PID: 18673", "st2actionrunner PID: 18675", "st2actionrunner PID: 18677", "st2actionrunner PID: 18679", "st2api PID: 87707", "st2api PID: 87772", "st2stream PID: 87833", "st2stream PID: 87852", "st2auth PID: 19594", "st2auth PID: 19665", "st2garbagecollector PID: 18848", "st2notifier PID: 18971", "st2resultstracker is not running.", "st2rulesengine PID: 19095", "st2sensorcontainer PID: 19219", "st2chatops is not running.", "st2timersengine PID: 19964", "st2workflowengine PID: 19843", "st2scheduler PID: 19720"]}
I work on fedora 32, python 3.8.6 with ansible-playbook in version 2.9.16. The target system is a CentOs 8.2, the virtual environment runs with python 3.6.8 and this is the content of pip freeze
Pip freeze
amqp==2.5.2
APScheduler==3.6.3
argcomplete==1.12.1
Babel==2.8.0
bcrypt==3.1.7
beautifulsoup4==4.9.3
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
colorama==0.4.4
cryptography==2.8
debtcollector==2.2.0
decorator==4.4.2
dnspython==1.16.0
eventlet==0.25.1
fasteners==0.15
flex==6.14.0
futurist==2.3.0
gitdb2==2.0.6
GitPython==2.1.15
greenlet==0.4.15
gunicorn==19.9.0
httplib2==0.18.1
idna==2.10
importlib-metadata==2.0.0
importlib-resources==3.0.0
ipaddr==2.2.0
iso8601==0.1.13
Jinja2==2.10.3
jsonpath-rw==1.4.0
jsonpointer==2.0
jsonschema==2.6.0
kazoo==2.8.0
kombu==4.6.6
linecache2==1.0.0
lockfile==0.12.2
logshipper==0.1.1.dev111
MarkupSafe==1.1.1
mock==2.0.0
mongoengine==0.18.2
monotonic==1.5
more-itertools==5.0.0
msgpack==1.0.0
netaddr==0.8.0
netifaces==0.10.9
networkx==1.11
nose==1.3.7
nose-parallel==0.3.1
nose-timer==0.7.5
ntlm-auth==1.5.0
orquesta==1.2.0
oslo.config==1.12.1
oslo.i18n==5.0.1
oslo.serialization==4.0.1
oslo.utils==3.37.0
paramiko==2.7.1
passlib==1.7.1
pbr==5.5.0
pika==1.1.0
ply==3.11
prettytable==1.0.1
prompt-toolkit==1.0.15
psutil==5.6.6
pycparser==2.20
pyinotify==0.9.6
pymongo==3.10.0
PyNaCl==1.4.0
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrabbit==1.1.0
python-dateutil==2.8.0
python-editor==1.0.4
python-gnupg==0.4.5
python-json-logger==2.0.1
python-statsd==2.1.0
pytz==2019.1
pywinrm==0.3.0
PyYAML==5.1.2
RandomWords==0.3.0
rednose==1.3.0
repoze.lru==0.7
requests==2.23.0
requests-ntlm==1.1.0
retrying==1.3.3
rfc3987==1.3.8
Routes==2.4.1
semver==2.9.0
simplejson==3.17.2
six==1.13.0
smmap==3.0.4
smmap2==3.0.1
soupsieve==2.0.1
sseclient-py==1.7
st2==3.3.0
st2-auth-backend-flat-file==0.1.0
st2-auth-backend-pam==0.1.0
st2actions==3.3.0
st2api==3.3.0
st2auth==3.3.0
st2client==3.3.0
st2common==3.3.0
st2debug==3.3.0
st2exporter==3.3.0
st2reactor==3.3.0
st2stream==3.3.0
st2tests==3.3.0
stackstorm-runner-action-chain==3.3.0
stackstorm-runner-announcement==3.3.0
stackstorm-runner-http==3.3.0
stackstorm-runner-inquirer==3.3.0
stackstorm-runner-local==3.3.0
stackstorm-runner-noop==3.3.0
stackstorm-runner-orquesta==3.3.0
stackstorm-runner-python==3.3.0
stackstorm-runner-remote==3.3.0
stackstorm-runner-winrm==3.3.0
stevedore==1.30.1
strict-rfc3339==0.7
tenacity==6.2.0
termstyle==0.1.11
tooz==1.66.1
traceback2==1.4.0
tzlocal==2.1
ujson==1.35
unittest2==1.1.0
urllib3==1.25.10
validate-email==1.3
vine==1.3.0
virtualenv==16.6.0
voluptuous==0.12.0
waitress==1.4.4
wcwidth==0.2.5
WebOb==1.8.5
WebTest==2.0.35
wrapt==1.12.1
xmltodict==0.12.0
yaql==1.1.3
zake==0.2.2
zipp==1.0.0
If I've missed any information please let me know. Thanks for your support.
Best regards
Marvin