ungleich-public/cdist
ungleich-public/cdist
7
4
Fork 10
Code Issues 58 Pull requests 15 Projects Releases Wiki Activity

cpu_cores does not work on amazon ec2 #205

New issue
Closed
opened 2021-11-20 15:21:26 +00:00 by ungleich-gitea · 5 comments
ungleich-gitea commented 2021-11-20 15:21:26 +00:00
Owner
Copy link

Created by: lloyd

cores="$(cat /proc/cpuinfo | grep "core id" | sort | uniq | wc -l)"

Yields incorrect results on some virtual machines. The results of this is 1 on an r3.xlarge instance, and I imagine for others too.

The issue is all cores have a core id of 0. removing uniq fixes the problem. Is there a system where cores are reported more than once? if not, could you just simplify to:

cores="$(cat /proc/cpuinfo | grep "core id" | wc -l)"
*Created by: lloyd* ``` cores="$(cat /proc/cpuinfo | grep "core id" | sort | uniq | wc -l)" ``` Yields incorrect results on some virtual machines. The results of this is `1` on an r3.xlarge instance, and I imagine for others too. The issue is all cores have a core id of 0. removing `uniq` fixes the problem. Is there a system where cores are reported more than once? if not, could you just simplify to: ``` cores="$(cat /proc/cpuinfo | grep "core id" | wc -l)" ```
ungleich-gitea added the
Stale
 label 2021-11-20 15:21:26 +00:00
ungleich-gitea commented 2021-11-20 15:47:54 +00:00
Author
Owner
Copy link

closed

closed
ungleich-gitea commented 2021-11-20 15:47:55 +00:00
Author
Owner
Copy link

Created by: telmich

Good point, dheule.

@lloyd what do you think about using your suggest version and create a new global explorer "cpu_cores_all" that includes cores based on hyperthreading?

*Created by: telmich* Good point, dheule. @lloyd what do you think about using your suggest version and create a new global explorer "cpu_cores_all" that includes cores based on hyperthreading?
ungleich-gitea commented 2021-11-20 15:47:57 +00:00
Author
Owner
Copy link

Created by: dheule

Hello Nico,

My intention to cpu_cores was and is to reflect the real physical cores
(without hyperthreading for example).

What about to make an additional explorer for example "cpu" which don't
reflect the real cores or sockets,
means only count the number of available cpus.

This is what counting the "core id" without uniq meens.

Regrards, Daniel

2014-05-21 23:49 GMT+02:00 Nico Schottelius notifications@github.com:

Hey lloyd,

I think this will break with hyperthreading:

[13:10] artist:~% grep "core id" /proc/cpuinfo
core id : 0
core id : 0
core id : 1
core id : 1

[on my notebook]

Maybe we should change cpu_cores to the version without uniq, like you
suggest and introduce another explorer "cpu_threads".

@dheule https://github.com/dheule and @astevenhttps://github.com/astevenwhat do you think about this change? In case we go for it, I'll make a
minor version bump -> incompatible change.

—
Reply to this email directly or view it on GitHubhttps://github.com/telmich/cdist/issues/337#issuecomment-43818840
.

*Created by: dheule* Hello Nico, My intention to cpu_cores was and is to reflect the real physical cores (without hyperthreading for example). What about to make an additional explorer for example "cpu" which don't reflect the real cores or sockets, means only count the number of available cpus. This is what counting the "core id" without uniq meens. Regrards, Daniel 2014-05-21 23:49 GMT+02:00 Nico Schottelius notifications@github.com: > Hey lloyd, > > I think this will break with hyperthreading: > > [13:10] artist:~% grep "core id" /proc/cpuinfo > core id : 0 > core id : 0 > core id : 1 > core id : 1 > > [on my notebook] > > Maybe we should change cpu_cores to the version without uniq, like you > suggest and introduce another explorer "cpu_threads". > > @dheule https://github.com/dheule and @astevenhttps://github.com/astevenwhat do you think about this change? In case we go for it, I'll make a > minor version bump -> incompatible change. > > — > Reply to this email directly or view it on GitHubhttps://github.com/telmich/cdist/issues/337#issuecomment-43818840 > .
ungleich-gitea commented 2021-11-20 15:47:58 +00:00
Author
Owner
Copy link

Created by: dheule

Hello lloyd,

seems that amazone ec2 has some courious cpu architecture.
Normaly on linux, the "core id" is only the same if hyperthreads are
connected to the same physical core.

2014-05-21 11:09 GMT+02:00 Lloyd Hilaiel notifications@github.com:

cores="$(cat /proc/cpuinfo | grep "core id" | sort | uniq | wc -l)"

Yields incorrect results on some virtual machines. The results of this is
1 on an r3.xlarge instance, and I imagine for others too.

The issue is all cores have a core id of 0. removing uniq fixes the
problem. Is there a system where cores are reported more than once? if not,
could you just simplify to:

cores="$(cat /proc/cpuinfo | grep "core id" | wc -l)"

—
Reply to this email directly or view it on GitHubhttps://github.com/telmich/cdist/issues/337
.

*Created by: dheule* Hello lloyd, seems that amazone ec2 has some courious cpu architecture. Normaly on linux, the "core id" is only the same if hyperthreads are connected to the same physical core. 2014-05-21 11:09 GMT+02:00 Lloyd Hilaiel notifications@github.com: > cores="$(cat /proc/cpuinfo | grep "core id" | sort | uniq | wc -l)" > > Yields incorrect results on some virtual machines. The results of this is > 1 on an r3.xlarge instance, and I imagine for others too. > > The issue is all cores have a core id of 0. removing uniq fixes the > problem. Is there a system where cores are reported more than once? if not, > could you just simplify to: > > cores="$(cat /proc/cpuinfo | grep "core id" | wc -l)" > > — > Reply to this email directly or view it on GitHubhttps://github.com/telmich/cdist/issues/337 > .
ungleich-gitea commented 2021-11-20 15:47:59 +00:00
Author
Owner
Copy link

Created by: telmich

Hey lloyd,

I think this will break with hyperthreading:

[13:10] artist:~% grep "core id" /proc/cpuinfo
core id : 0
core id : 0
core id : 1
core id : 1

[on my notebook]

Maybe we should change cpu_cores to the version without uniq, like you suggest and introduce another explorer "cpu_threads".

@dheule and @asteven what do you think about this change? In case we go for it, I'll make a minor version bump -> incompatible change.

*Created by: telmich* Hey lloyd, I think this will break with hyperthreading: [13:10] artist:~% grep "core id" /proc/cpuinfo core id : 0 core id : 0 core id : 1 core id : 1 [on my notebook] Maybe we should change cpu_cores to the version without uniq, like you suggest and introduce another explorer "cpu_threads". @dheule and @asteven what do you think about this change? In case we go for it, I'll make a minor version bump -> incompatible change.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
7.0
6.9
py3.10
ander/__package_apt_update_index
haproxy-dualstack
ander/__sed
beta
ander/os_version_debian_sid
evilham-compatibility-fixes
ander/__rsync
__snakeoil_cert
ander/update_readme
__download_improvements
feature/onchange
cleanup/string-formatting
feature/type-relationship-graph
cleanup/ssh-auth-keys-types
__letsencrypt_cert-fix-hooks
bugfix/preos-debug
bugfix/in-script-import
6.8
bugfix/sphinx-docs-build
6.7
cherry-pick-2f433a14
bugfix/make-code-consistent
6.6
regain-py3.2-support
6.5
bugfix/multiple-log-lines
matterbridge
coturn
alpinefix
matrix
new-type/network-interface
feature/process
6.4
feature/info-command
feature/libexec
6.3
preos-plugins-dir-opt
gitlab-ci
6.2
order-dep-fix
6.1
6.0
build/support-pip-from-git
feature/shell-lib
5.1
feature/support-type-deprecation
5.0
feature/python-types
4.11
4.10
shellcheck
4.9
4.8
freebsd-improvements
new-prometheus
key_value-onchange
feature/output_streams
AnotherKamila-patch-1
__letsencrypt_cert-fixes
letsencrypt-cron-fix
4.7
newtype-__letsencrypt_cert
os_explorer_devuan_fix
prometheus-exporter-fixes
daemontools-for-fbsd
type-prometheus-exporter-from-master
prometheus-more-fixes
4.6
4.5
fix-j
steven-backport
4.4
prometheus-fixes
grafana_dashboard
prometheus
daemontools
consul_improvements
feature/trigger
4.3
4.0-pre-not-stable
4.2
4.1
4.0
feature_install_and_preos
3.1
no-dot-cdist
random_dot_cdist
feature_yum_url
feature_files_export
3.0
2.3
2.2
2.1
ssh_callback
2.0
archive_shell_function_approach
1.7
1.6
1.5
1.4
1.3
1.2
1.1
1.0
7.0.0
6.9.8
6.9.7
6.9.6
6.9.5
6.9.4
6.9.3
6.9.2
6.9.1
6.9.0
6.8.0
6.7.0
6.6.0
6.5.6
6.5.5
6.5.4
6.5.3
6.5.2
6.5.1
6.5.0
6.4.0
6.3.0
6.2.0
6.1.1
6.1.0
6.0.4
6.0.3
6.0.2
6.0.1
6.0.0
5.1.3
5.1.2
5.1.1
5.1.0
5.0.2
5.0.1
5.0.0
4.11.1
4.11.0
4.10.11
4.10.10
4.10.9
4.10.8
4.10.7
4.10.6
4.10.5
4.10.4
4.10.3
4.10.2
4.10.1
4.10.0
4.9.1
4.9.0
4.8.4
4.8.3
4.8.2
4.8.1
4.8.0
4.7.3
4.7.2
4.7.1
4.7.0
4.6.1
4.6.0
4.5.0
4.4.4
4.4.3
4.4.2
4.4.1
4.4.0
4.3.2
4.3.1
4.3.0
4.2.2
4.2.1
4.2.0
4.1.0
4.0.0
3.1.13
tn1
3.1.12
3.1.11
3.1.10
3.1.9
3.1.8
3.1.7
3.1.6
4.0.0pre3
3.1.5
3.1.4
3.1.3
3.1.2
3.1.1
3.1.0
4.0.0pre2
3.0.9
3.0.8
3.0.7
3.0.6
3.0.5
3.0.4
3.0.3
4.0.0pre1
3.0.2
3.0.1
3.0.0
2.3.7
2.3.6
2.3.5
2.3.4
2.3.3
2.3.2
2.3.1
2.3.0
2.2.0
2.1.2
2.1.1
2.1.0
2.1.0pre8
2.1.0pre7
2.1.0pre6
2.0.15
2.1.0pre5
2.1.0pre4
2.1.0pre3
2.1.0pre2
2.1.0pre1
2.0.14
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.7.1
1.7.0
1.6.2
1.6.1
1.6.0
1.5.0
1.4.1
1.4.0
1.3.2
1.3.1
1.3.0
1.2.0
1.1.0
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.9.9
0.9.8
0.9.7
0.9.6
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Labels
Clear labels   
bugfix

   
cleanup

   
discussion

   
documentation

   
doing

   
done

   
feature

   
improvement

   
packaging

   
Stale

   
testing

   
TODO
No labels
bugfix
cleanup
discussion
documentation
doing
done
feature
improvement
packaging
Stale
testing
TODO
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: ungleich-public/cdist#205
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?