You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 16:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4.osadl.org (updated Sat Apr 27, 2024 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1026769993634,1cyclictest966606-21kworker/0:2+pm19:35:320
1026769993025,3cyclictest260-21systemd-journal00:35:000
1026772992621,3cyclictest1377852-21apt23:40:001
1026769992420,3cyclictest1438625-21cron00:20:000
1026778992319,2cyclictest0-21swapper/319:55:003
1026776992320,2cyclictest1052893-21idleruntime-cro19:35:012
1026772992320,2cyclictest1213114-21idleruntime-cro21:50:001
1026772992319,3cyclictest1396130-21kthreadcore23:50:181
1026769992318,3cyclictest260-21systemd-journal21:29:590
1026778992219,2cyclictest1213108-21cron21:50:003
1026778992117,3cyclictest1358248-21kthreadcore23:25:203
1026778992017,2cyclictest1147965-21idleruntime-cro21:05:003
1026776992017,2cyclictest1221474-21irqcore21:55:162
1026772992016,3cyclictest1236430-21irqcore22:05:201
1026772992015,3cyclictest0-21swapper/123:10:011
1026769992012,6cyclictest1236198-21fw_conntrack22:05:170
1026776991916,2cyclictest1296208-21tr22:45:112
1026776991916,2cyclictest0-21swapper/200:15:002
1026772991914,3cyclictest2955-21cron20:59:591
1026769991916,2cyclictest1289160-21irqcore22:40:170
1026769991913,2cyclictest60650irq/128-enp0s29f1:rx-021:34:470
1026778991815,2cyclictest1447572-21kthreadcore00:25:183
1026778991815,2cyclictest1326481-21irqcore23:05:173
1026776991815,2cyclictest1137440-21munin-run20:55:002
1026776991814,3cyclictest1213603-21cpuspeed_turbos21:50:152
1026772991814,3cyclictest1453511-21sh00:30:001
1026772991814,3cyclictest0-21swapper/121:20:001
1026772991814,2cyclictest1176542-21irqcore21:25:181
1026769991815,2cyclictest260-21systemd-journal22:25:010
1026769991815,2cyclictest1090444-21irqcore20:10:140
1026769991815,2cyclictest1084400-21sh20:05:000
1026769991813,2cyclictest60650irq/128-enp0s29f1:rx-023:45:530
1026769991813,2cyclictest60650irq/128-enp0s29f1:rx-023:31:580
1026769991813,2cyclictest60650irq/128-enp0s29f1:rx-021:18:580
102677899179,6cyclictest1303205-21cron22:50:013
1026778991714,2cyclictest1408302-21cron23:59:593
1026778991713,3cyclictest1169074-21irqcore21:20:203
1026778991710,5cyclictest260-21systemd-journal21:39:593
1026776991714,2cyclictest1468525-21sh00:40:002
1026776991713,2cyclictest0-21swapper/200:30:002
1026772991714,2cyclictest1190701-21ssh21:35:011
102676999178,6cyclictest260-21systemd-journal23:20:000
102676999174,3cyclictest11131162kthreadcore20:30:160
102676999174,1cyclictest14462731latency_hist00:25:000
1026769991714,2cyclictest61450irq/129-enp0s29f1:tx-021:45:160
1026769991714,2cyclictest60650irq/128-enp0s29f1:rx-022:54:000
1026769991714,2cyclictest60650irq/128-enp0s29f1:rx-021:50:090
1026769991714,2cyclictest60650irq/128-enp0s29f1:rx-019:55:200
1026769991714,2cyclictest60650irq/128-enp0s29f1:rx-000:26:580
1026769991714,2cyclictest60650irq/128-enp0s29f1:rx-000:10:340
1026769991714,2cyclictest171ktimers/023:38:180
1026769991713,3cyclictest1300774-21unixbench_singl22:45:260
1026769991712,3cyclictest60650irq/128-enp0s29f1:rx-023:05:110
1026769991712,2cyclictest60650irq/128-enp0s29f1:rx-000:38:580
1026769991711,4cyclictest61450irq/129-enp0s29f1:tx-022:30:210
1026778991613,2cyclictest1367404-21needreboot23:30:213
1026778991613,2cyclictest1250507-21idleruntime-cro22:14:593
1026778991613,2cyclictest1057736-21fwupd19:35:313
1026778991613,2cyclictest0-21swapper/322:15:153
1026778991612,2cyclictest0-21swapper/323:45:143
1026776991613,2cyclictest1161646-21grep21:15:252
1026776991612,2cyclictest0-21swapper/223:05:172
1026776991611,4cyclictest1360519-21ssh23:25:292
1026772991613,2cyclictest1251971-21kthreadcore22:15:151
1026772991613,2cyclictest1154135-21irqcore21:10:181
1026772991613,2cyclictest1094940-21idleruntime-cro20:15:001
1026772991612,3cyclictest311ktimers/123:05:011
1026772991612,3cyclictest1408945-21fschecks_time00:00:141
102676999167,4cyclictest0-21swapper/019:20:180
102676999164,2cyclictest12550502modprobe22:15:190
102676999162,8cyclictest191rcu_preempt23:50:180
1026769991614,1cyclictest60650irq/128-enp0s29f1:rx-023:25:200
1026769991614,1cyclictest60650irq/128-enp0s29f1:rx-023:10:360
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-023:55:160
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-023:40:560
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-023:20:030
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-023:00:160
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-022:25:270
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-022:01:360
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-021:55:160
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-021:36:480
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-021:11:440
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-020:57:250
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-020:53:510
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-020:35:170
1026769991613,2cyclictest60650irq/128-enp0s29f1:rx-000:07:260
1026769991612,3cyclictest1243421-21cpuspeed_turbos22:10:120
1026769991611,3cyclictest1105906-21munin-plugin-st20:25:010
102677899158,5cyclictest1433643-21cut00:15:173
1026778991512,2cyclictest1346275-21sshd23:16:473
1026778991512,2cyclictest1099286-21needreboot20:15:193
1026778991512,2cyclictest0-21swapper/300:40:003
1026778991512,2cyclictest0-21swapper/300:20:253
1026778991511,3cyclictest1032709-21irqcore19:15:163
102677699159,4cyclictest497-21dbus-daemon22:25:012
102677699158,6cyclictest1042434-21munin-run19:25:012
102677699153,3cyclictest3204197-21kworker/u8:2+xprtiod22:29:142
1026776991513,1cyclictest401ktimers/223:00:262
1026776991512,2cyclictest1063371-21cron19:45:002
1026776991511,3cyclictest1187688-21needreboot21:30:212
1026776991511,3cyclictest1032517-21gltestperf19:15:142
102677299152,3cyclictest1288631-21apt-get22:40:111
1026772991512,2cyclictest1191147-21cpuspeed_turbos21:35:141
1026772991512,2cyclictest0-21swapper/121:04:591
1026772991511,3cyclictest1079718-21head20:00:151
1026772991511,3cyclictest1078673-21unixbench_multi19:55:281
1026772991511,3cyclictest1047236-21unixbench_singl19:25:251
1026772991511,3cyclictest1047236-21unixbench_singl19:25:251
102676999159,2cyclictest60650irq/128-enp0s29f1:rx-022:35:380
1026769991513,1cyclictest60650irq/128-enp0s29f1:rx-020:43:420
1026769991512,2cyclictest60650irq/128-enp0s29f1:rx-022:55:140
1026769991512,2cyclictest60650irq/128-enp0s29f1:rx-021:20:070
1026769991512,2cyclictest60650irq/128-enp0s29f1:rx-019:50:340
1026769991512,2cyclictest60650irq/128-enp0s29f1:rx-019:25:280
1026769991512,2cyclictest60650irq/128-enp0s29f1:rx-019:25:270
1026769991512,2cyclictest1089218-21unixbench_multi20:05:250
1026769991511,3cyclictest61450irq/129-enp0s29f1:tx-019:30:210
102677899149,3cyclictest1355432-21turbostat.cron23:25:013
102677899149,3cyclictest1153229-21turbostat21:09:593
102677899149,3cyclictest0-21swapper/321:55:003
102677899148,4cyclictest1177544-21cut21:25:203
102677899147,5cyclictest1190675-21latency_hist21:35:013
102677899144,2cyclictest0-21swapper/321:40:253
1026778991412,1cyclictest3008581-21kworker/3:3+events_power_efficient00:01:083
1026778991411,2cyclictest1430930-21latency_hist00:14:593
1026778991411,2cyclictest1338033-21unixbench_multi23:10:283
1026778991411,2cyclictest1236429-21sed22:05:203
1026778991411,2cyclictest1122056-21cpuspeed_turbos20:40:113
1026778991411,2cyclictest1074288-21cpuspeed_turbos19:55:153
1026778991411,2cyclictest0-21swapper/322:55:013
1026778991411,1cyclictest0-21swapper/323:35:193
1026778991410,3cyclictest1293342-21unixbench_singl22:40:263
1026778991410,2cyclictest0-21swapper/322:20:253
1026778991410,2cyclictest0-21swapper/322:00:133
102677699149,4cyclictest523-21gdbus23:35:012
102677699148,5cyclictest1337673-21ntp_states23:10:242
102677699148,4cyclictest260-21systemd-journal00:03:472
1026776991412,1cyclictest1026764-21cyclictest19:50:212
1026776991411,2cyclictest61650irq/133-enp0s29f1:tx-220:44:472
1026776991411,2cyclictest1280809-21cron22:35:002
1026776991411,2cyclictest1237038-21kthreadcore22:05:212
1026776991410,3cyclictest1348657-21gltestperf23:20:172
1026776991410,3cyclictest1311126-21cpuspeed_turbos22:55:132
1026776991410,3cyclictest1131711-21unixbench_singl20:45:292
1026776991410,2cyclictest0-21swapper/219:35:132
102677299148,4cyclictest1387198-21cut23:45:171
102677299148,4cyclictest0-21swapper/123:55:131
102677299147,6cyclictest1417575-21cut00:05:171
102677299145,2cyclictest1068646-21munin-run19:50:001
102677299141,5cyclictest1026698-21latency_hist19:10:001
1026772991413,1cyclictest311ktimers/121:21:191
1026772991411,2cyclictest1461446-21sh00:35:121
1026772991411,2cyclictest1258432-21cpuspeed_turbos22:20:121
1026772991411,2cyclictest1229546-21kthreadcore22:00:181
1026772991410,3cyclictest1273778-21cpuspeed_turbos22:30:131
1026772991410,2cyclictest0-21swapper/123:45:001
1026772991410,2cyclictest0-21swapper/123:30:211
1026772991410,2cyclictest0-21swapper/122:10:161
1026772991410,2cyclictest0-21swapper/100:30:141
102676999148,1cyclictest191rcu_preempt19:45:160
102676999147,5cyclictest260-21systemd-journal19:15:000
1026769991413,1cyclictest60650irq/128-enp0s29f1:rx-021:00:190
1026769991411,2cyclictest60650irq/128-enp0s29f1:rx-019:15:240
102677899139,2cyclictest0-21swapper/320:20:153
102677899137,4cyclictest497-21dbus-daemon22:40:013
1026778991311,1cyclictest1382405-21needreboot23:40:223
1026778991310,2cyclictest491ktimers/321:14:373
1026778991310,2cyclictest0-21swapper/322:25:003
1026778991310,2cyclictest0-21swapper/321:20:003
1026778991310,2cyclictest0-21swapper/319:50:013
102677699139,3cyclictest994130-21munin-node20:20:012
102677699139,2cyclictest0-21swapper/200:20:002
102677699138,1cyclictest10451021kthreadcore19:25:182
102677699138,1cyclictest10451021kthreadcore19:25:182
102677699137,4cyclictest1099938-21sshd20:16:482
102677699137,4cyclictest1079135-21cron20:00:002
102677699133,3cyclictest1417264-21kthreadcore00:05:162
102677699133,3cyclictest1203382-21ssh21:40:442
102677699133,2cyclictest391rcuc/220:05:162
102677699131,2cyclictest13070162cut22:50:242
1026776991311,1cyclictest1153227-21diskmemload21:20:192
1026776991311,1cyclictest1109648-21kthreadcore20:25:202
1026776991310,2cyclictest1453961-21cpuspeed_turbos00:30:122
1026776991310,2cyclictest0-21swapper/221:25:162
1026776991310,2cyclictest0-21swapper/220:59:592
102677299139,3cyclictest1341050-21fschecks_count23:15:131
102677299139,3cyclictest1148524-21gltestperf21:05:181
102677299139,3cyclictest1100330-21apt-key20:20:001
102677299139,3cyclictest0-21swapper/122:25:001
102677299139,2cyclictest0-21swapper/122:40:011
102677299137,4cyclictest260-21systemd-journal19:41:481
102677299137,4cyclictest1438502-21sshd00:19:471
1026772991311,2cyclictest1420317-21kworker/u8:2+xprtiod00:21:591
1026772991310,2cyclictest484-21rpcbind00:15:001
1026772991310,2cyclictest1296396-21sh22:45:131
1026772991310,2cyclictest1210290-21kworker/u8:3+xprtiod21:50:371
102677299130,1cyclictest1105876-21cat20:24:561
102676999139,3cyclictest0-21swapper/021:10:000
1026769991312,1cyclictest60650irq/128-enp0s29f1:rx-020:25:210
1026769991310,2cyclictest60650irq/128-enp0s29f1:rx-020:15:220
102677899129,2cyclictest1311410-21gltestperf22:55:163
102677899129,1cyclictest0-21swapper/319:40:163
102677899127,2cyclictest1132149-21munin-plugin-st20:50:003
102677899125,6cyclictest1084775-21cut20:05:103
102677899125,5cyclictest497-21dbus-daemon20:40:003
102677899121,4cyclictest0-21swapper/300:05:173
102677899121,10cyclictest1225263-21proc_pri21:55:233
1026778991210,1cyclictest509-21systemd-logind19:35:013
102677899120,4cyclictest1453829-21/usr/sbin/munin00:30:283
102677699129,2cyclictest3204197-21kworker/u8:2+xprtiod21:35:172
102677699129,2cyclictest1908575-21kworker/u8:0+xprtiod22:10:492
102677699129,2cyclictest1401225-21cpuspeed_turbos23:55:112
102677699129,2cyclictest1386975-21kthreadcore23:45:172
102677699129,2cyclictest1330174-21kworker/u8:1+xprtiod23:15:322
102677699129,2cyclictest0-21swapper/223:35:132
102677699128,3cyclictest1210290-21kworker/u8:3+xprtiod22:36:082
102677699128,3cyclictest1153700-21cpuspeed_turbos21:10:132
102677699128,3cyclictest1111482-21cpuspeed_turbos20:30:112
102677699128,2cyclictest2955-21cron21:10:002
102677699126,5cyclictest1394853-21tr23:50:172
102677699125,5cyclictest1094939-21latency_hist20:15:002
102677699125,1cyclictest12292111irqcore22:00:172
1026776991210,2cyclictest1330174-21kworker/u8:1+xprtiod23:42:532
102677699120,4cyclictest1031964-21latency_hist19:15:002
102677299129,2cyclictest311ktimers/123:29:571
102677299129,2cyclictest311ktimers/121:40:461
102677299129,2cyclictest1116453-21systemd-run20:35:001
102677299129,2cyclictest1106363-21fschecks_count20:25:141
102677299129,2cyclictest1069015-21cpuspeed_turbos19:50:101
102677299128,2cyclictest0-21swapper/123:10:161
102677299125,4cyclictest1037448-21apt-get19:20:111
1026772991210,2cyclictest3204197-21kworker/u8:2+xprtiod22:54:071
1026772991210,2cyclictest1210290-21kworker/u8:3+xprtiod21:55:211
102677899118,2cyclictest1106231-21cpuspeed_turbos20:25:133
102677899117,3cyclictest509-21systemd-logind20:12:103
102677899117,3cyclictest509-21systemd-logind19:26:303
102677899117,3cyclictest509-21systemd-logind19:26:303
102677899117,3cyclictest1115617-21nvmesmart_nvme020:30:213
102677899115,4cyclictest1278606-21ssh22:30:483
102677899115,4cyclictest1136666-21wc20:50:243
102677899111,4cyclictest0-21swapper/323:50:133
102677899110,2cyclictest0-21swapper/319:22:293
102677699119,2cyclictest1210290-21kworker/u8:3+xprtiod22:15:022
102677699118,2cyclictest1143263-21gltestperf21:00:142
102677699118,2cyclictest0-21swapper/219:45:132
102677699117,3cyclictest4050ktimers/221:45:302
102677299118,2cyclictest0-21swapper/120:40:231
102677299118,2cyclictest0-21swapper/120:10:011
102677299117,3cyclictest1318021-21ssh22:59:481
102677299117,2cyclictest0-21swapper/120:40:011
102677299115,4cyclictest1350906-21tr23:20:211
102677299114,5cyclictest260-21systemd-journal19:40:001
102677299113,3cyclictest1037174-21idleruntime-cro19:20:001
102677299110,9cyclictest1031979-21awk19:15:001
102676999119,1cyclictest211rcuc/019:10:000
102676999116,3cyclictest1058886-21sort19:40:190
102677899107,2cyclictest516-21rs:main3
102677899107,2cyclictest1027139-21df19:10:143
102677899101,7cyclictest1080500-21kthreadcore20:00:193
102677899101,2cyclictest191rcu_preempt23:00:173
102677699100,2cyclictest0-21swapper/220:00:272
102677299102,3cyclictest1132161-21basename20:49:591
102677299101,7cyclictest1049014-21kthreadcore19:30:181
102677299101,2cyclictest260-21systemd-journal20:55:001
102676999100,3cyclictest1127561-21gltestperf20:45:150
10267769994,4cyclictest1116955-21perl20:35:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional