You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 11:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Sat Jun 20, 2026 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
528421280,10sleep15285-21ssh22:43:451
31993991172,70cyclictest17275-21ssh22:09:040
31827211214,65sleep131860-21munin-plugin-st18:58:441
31827211214,65sleep131860-21munin-plugin-st18:58:441
319979910257,33cyclictest32642-21latency_hist20:38:431
31997991004,76cyclictest26703-21apt-get00:14:011
31997991004,76cyclictest26703-21apt-get00:14:011
3199799993,67cyclictest17265-21ssh22:09:031
31993999742,24cyclictest31981-21cyclictest21:39:480
31993999742,24cyclictest31981-21cyclictest21:39:480
3199399962,74cyclictest13711-21kworker/u19:1+ext4-rsv-conversion00:14:110
3199399962,74cyclictest13711-21kworker/u19:1+ext4-rsv-conversion00:14:110
31997999462,20cyclictest23446-21latency_hist00:08:471
32000999213,34cyclictest0-21swapper/200:00:282
31993999254,11cyclictest181rcu_preempt22:41:420
32000999171,12cyclictest11490-21apt-get22:53:532
32000999028,50cyclictest29292-21sshd23:23:582
32000999019,29cyclictest0-21swapper/223:33:492
32000999019,29cyclictest0-21swapper/223:33:492
31993999063,8cyclictest181rcu_preempt20:49:050
31993999063,19cyclictest7283-21sh20:59:060
32000998920,27cyclictest24086-21sed00:09:112
31997998967,11cyclictest0-21swapper/122:53:461
31997998966,14cyclictest9901-21cpuspeed19:29:001
31997998965,16cyclictest2374-21systemd-journal22:20:591
31993998953,30cyclictest17138-21latency_hist22:08:480
32000998824,27cyclictest14197-21tail21:09:192
32000998822,27cyclictest19676-21snmpd20:34:182
31997998864,15cyclictest10276-21kworker/u17:2-rpciod21:19:141
3199799882,63cyclictest8278-21ssh21:54:291
3199799882,63cyclictest8278-21ssh21:54:291
32000998766,15cyclictest9952-21apt-key21:03:472
31993998742,34cyclictest0-21swapper/020:03:430
32000998665,14cyclictest1916-21latency_hist20:43:462
32000998624,27cyclictest2768-21runrttasks22:39:002
31997998662,17cyclictest28841-21latency_hist22:28:451
31997998655,8cyclictest181rcu_preempt00:12:341
31997998649,12cyclictest181rcu_preempt22:04:001
31997998639,13cyclictest181rcu_preempt22:59:401
31997998639,13cyclictest181rcu_preempt22:59:401
31997998625,27cyclictest0-21swapper/122:54:121
31993998658,19cyclictest6657-21diskmemload23:36:390
31993998658,19cyclictest6657-21diskmemload23:36:390
32000998513,38cyclictest11376-40mandb23:48:472
31993998554,8cyclictest181rcu_preempt19:44:070
32000998423,26cyclictest352-21runrttasks22:09:022
32000998410,37cyclictest11161-21ssh22:53:422
31997998460,17cyclictest1-21systemd20:03:441
31993998444,9cyclictest181rcu_preempt23:30:250
32000998321,23cyclictest3538-21ssh21:45:562
32000998318,24cyclictest0-21swapper/221:33:152
32000998316,37cyclictest9209-21ssh21:55:312
32000998316,37cyclictest9209-21ssh21:55:312
31997998364,9cyclictest0-21swapper/120:19:181
31997998359,9cyclictest31959-21ssh21:39:141
31997998359,9cyclictest31959-21ssh21:39:141
31997998345,11cyclictest181rcu_preempt20:08:561
31997998345,11cyclictest181rcu_preempt20:08:561
31997998328,25cyclictest31981-21cyclictest00:23:431
31993998356,19cyclictest23843-21sh23:14:100
32000998259,16cyclictest31935-21latency_hist22:33:472
32000998223,27cyclictest22769-21runrttasks21:24:152
32000998222,34cyclictest30514-21ssh21:37:202
32000998221,26cyclictest352-21runrttasks23:31:012
32000998220,25cyclictest7212-21taskset19:19:182
32000998220,25cyclictest7212-21taskset19:19:182
32000998216,36cyclictest352-21runrttasks22:01:472
31997998260,12cyclictest0-21swapper/119:09:181
31997998237,15cyclictest181rcu_preempt22:16:161
31993998249,9cyclictest181rcu_preempt00:23:390
32000998163,11cyclictest17460-21sshd23:57:592
32000998163,11cyclictest17460-21sshd23:57:592
32000998122,23cyclictest18513-21seq21:17:502
32000998122,23cyclictest18513-21seq21:17:502
32000998119,30cyclictest20864-21basename20:03:432
32000998113,29cyclictest23729-21ntp_states22:19:112
32000998111,36cyclictest0-21swapper/220:54:352
31997998141,8cyclictest181rcu_preempt20:48:591
31997998136,11cyclictest181rcu_preempt20:58:461
31997998128,21cyclictest0-21swapper/122:02:351
3199399814,62cyclictest8290-21ssh21:54:310
3199399814,62cyclictest8290-21ssh21:54:310
31993998132,8cyclictest181rcu_preempt23:20:000
31993998126,25cyclictest0-21swapper/019:43:460
32000998041,28cyclictest22181-21timerandwakeup20:04:452
32000998041,28cyclictest22181-21timerandwakeup20:04:452
32000998031,15cyclictest0-21swapper/222:34:072
32000998031,15cyclictest0-21swapper/222:34:072
32000998020,26cyclictest6657-21diskmemload22:44:332
32000998020,26cyclictest6657-21diskmemload22:44:332
32000998020,24cyclictest6164-21seq19:16:442
32000998019,24cyclictest0-21swapper/200:14:002
32000998019,24cyclictest0-21swapper/200:14:002
31997998065,10cyclictest31981-21cyclictest21:44:011
31997998060,14cyclictest17706-21ssh23:59:011
31997998044,11cyclictest181rcu_preempt20:44:061
31997998041,7cyclictest181rcu_preempt23:14:231
3199399804,64cyclictest10188-21apt-get21:04:070
3199399804,64cyclictest10188-21apt-get21:04:070
31993998038,10cyclictest181rcu_preempt22:28:480
31993998033,8cyclictest181rcu_preempt19:54:190
31993998033,8cyclictest181rcu_preempt19:54:190
32000997956,16cyclictest23694-21cut23:14:062
32000997925,23cyclictest0-21swapper/200:04:122
32000997921,22cyclictest31041-21ssh00:20:102
32000997920,23cyclictest19158-21seq19:57:432
32000997920,23cyclictest19158-21seq19:57:432
32000997920,19cyclictest27609-21seq23:19:552
32000997919,34cyclictest12233-21ssh21:06:592
32000997919,34cyclictest12233-21ssh21:06:592
32000997919,23cyclictest2916-21seq19:06:362
32000997918,24cyclictest4575-21seq19:11:542
32000997915,43cyclictest352-21runrttasks20:24:412
31997997954,8cyclictest181rcu_preempt19:14:011
31997997954,18cyclictest1-21systemd21:48:591
31997997954,18cyclictest1-21systemd21:48:591
31997997951,19cyclictest13669-21latency21:09:081
31997997950,14cyclictest31348-21ssh23:27:091
31997997943,8cyclictest181rcu_preempt19:43:551
31993997951,11cyclictest7985-21latency_hist21:53:470
31993997943,7cyclictest181rcu_preempt19:29:170
31993997938,32cyclictest2266-21modprobe22:38:420
31993997938,32cyclictest2266-21modprobe22:38:420
31993997935,28cyclictest181rcu_preempt23:03:520
3186227913,41sleep231858-21munin-run18:58:452
3186227913,41sleep231858-21munin-run18:58:452
32000997858,14cyclictest352-21runrttasks19:29:202
32000997819,22cyclictest6352-21ssh21:50:222
32000997819,22cyclictest6352-21ssh21:50:222
32000997817,25cyclictest17407-21seq19:50:512
32000997817,25cyclictest17407-21seq19:50:512
32000997813,23cyclictest17090-21latency_hist23:03:462
32000997813,23cyclictest17090-21latency_hist23:03:462
31997997858,14cyclictest22854-21latency_hist20:08:461
31997997858,14cyclictest22854-21latency_hist20:08:461
31997997830,12cyclictest181rcu_preempt22:30:461
31997997827,23cyclictest0-21swapper/119:06:241
31997997825,12cyclictest181rcu_preempt23:04:201
31993997856,8cyclictest181rcu_preempt22:13:540
31993997850,8cyclictest181rcu_preempt19:03:450
31993997842,11cyclictest58-21kcompactd021:30:100
31993997834,8cyclictest181rcu_preempt21:59:520
31993997832,10cyclictest181rcu_preempt20:27:090
31993997817,26cyclictest17644-21apt19:53:430
31993997817,26cyclictest17644-21apt19:53:430
32000997720,21cyclictest2374-21systemd-journal23:08:442
31997997758,10cyclictest0-21swapper/122:34:121
31997997758,10cyclictest0-21swapper/122:34:121
31997997748,20cyclictest6657-21diskmemload22:48:461
31997997748,20cyclictest6657-21diskmemload22:48:461
3199799772,38cyclictest28352-21apt-get21:34:011
31993997746,9cyclictest181rcu_preempt21:23:350
31993997742,13cyclictest181rcu_preempt23:53:460
31993997734,8cyclictest181rcu_preempt20:34:010
31993997733,8cyclictest181rcu_preempt20:54:140
31993997730,8cyclictest181rcu_preempt23:59:180
31993997727,7cyclictest181rcu_preempt22:54:020
31993997719,29cyclictest0-21swapper/021:44:060
31993997717,36cyclictest15-21ksoftirqd/020:38:530
31993997715,23cyclictest161ktimers/000:24:040
32000997617,26cyclictest28027-21seq22:26:462
32000997615,27cyclictest0-21swapper/222:03:552
32000997613,27cyclictest0-21swapper/220:49:062
32000997612,25cyclictest20793-21/usr/sbin/munin22:14:142
31997997647,10cyclictest181rcu_preempt23:32:181
31997997645,11cyclictest181rcu_preempt21:14:181
31997997645,11cyclictest181rcu_preempt21:14:181
31997997639,8cyclictest181rcu_preempt21:04:221
31997997639,8cyclictest181rcu_preempt21:04:221
31997997639,12cyclictest181rcu_preempt21:29:461
3199799763,30cyclictest9009-21perl23:44:061
31993997659,11cyclictest6657-21diskmemload21:09:080
31993997646,9cyclictest181rcu_preempt21:28:440
31993997646,13cyclictest181rcu_preempt21:49:140
31993997644,10cyclictest181rcu_preempt22:49:490
31993997637,8cyclictest181rcu_preempt20:14:160
31993997629,8cyclictest181rcu_preempt21:14:190
31993997629,8cyclictest181rcu_preempt21:14:190
32000997520,23cyclictest24109-21seq20:10:522
32000997520,23cyclictest24109-21seq20:10:522
31997997555,10cyclictest0-21swapper/120:33:461
31997997541,7cyclictest181rcu_preempt23:39:081
31997997530,24cyclictest0-21swapper/119:51:091
31997997530,24cyclictest0-21swapper/119:51:091
31997997526,24cyclictest0-21swapper/120:40:071
31993997553,8cyclictest181rcu_preempt23:57:090
31993997553,8cyclictest181rcu_preempt23:57:090
31993997552,16cyclictest0-21swapper/023:48:450
31993997546,12cyclictest181rcu_preempt19:19:190
31993997546,12cyclictest181rcu_preempt19:19:190
31993997545,10cyclictest181rcu_preempt19:14:160
31993997535,8cyclictest181rcu_preempt23:27:220
31993997534,18cyclictest0-21swapper/019:34:080
31993997520,35cyclictest0-21swapper/022:44:590
31993997520,35cyclictest0-21swapper/022:44:590
31993997515,8cyclictest0-21swapper/000:12:160
31993997515,20cyclictest6657-21diskmemload23:39:040
32000997420,23cyclictest27511-21seq20:23:242
32000997419,23cyclictest9502-21seq19:28:232
32000997419,23cyclictest9502-21seq19:28:232
32000997418,25cyclictest12749-21seq19:38:272
32000997416,36cyclictest352-21runrttasks19:40:062
32000997416,24cyclictest8285-21chrt23:43:322
31997997453,12cyclictest0-21swapper/119:54:231
31997997453,12cyclictest0-21swapper/119:54:231
31997997448,10cyclictest181rcu_preempt00:27:511
31997997435,19cyclictest240-21rpcbind23:09:141
31997997428,9cyclictest181rcu_preempt19:34:191
31997997428,18cyclictest0-21swapper/120:24:041
31993997461,7cyclictest0-21swapper/023:08:530
31993997449,9cyclictest181rcu_preempt20:44:160
31993997435,8cyclictest181rcu_preempt22:24:440
3199399743,44cyclictest15-21ksoftirqd/022:18:560
31993997431,10cyclictest181rcu_preempt00:04:180
32000997319,24cyclictest22797-21seq23:13:022
32000997315,25cyclictest0-21swapper/221:42:562
32000997315,25cyclictest0-21swapper/221:42:562
32000997313,25cyclictest0-21swapper/220:14:102
32000997312,28cyclictest333-21ssh00:24:022
31997997357,10cyclictest31981-21cyclictest21:25:351
31997997351,14cyclictest3357-21ntp_offset23:34:111
31997997351,14cyclictest3357-21ntp_offset23:34:111
31997997341,8cyclictest181rcu_preempt19:18:551
31997997341,8cyclictest181rcu_preempt19:18:551
31997997334,19cyclictest1299-21munin-node23:54:201
31997997334,19cyclictest1299-21munin-node23:54:201
31997997331,17cyclictest0-21swapper/119:40:281
31997997326,21cyclictest0-21swapper/123:19:151
31993997337,8cyclictest181rcu_preempt21:37:430
31993997331,9cyclictest0-21swapper/020:04:300
31993997331,9cyclictest0-21swapper/020:04:300
32000997215,27cyclictest0-21swapper/219:00:362
32000997215,22cyclictest0-21swapper/220:46:442
31997997256,9cyclictest0-21swapper/121:03:451
31997997237,10cyclictest181rcu_preempt18:58:581
31997997233,9cyclictest181rcu_preempt20:29:331
31993997246,10cyclictest181rcu_preempt20:09:120
31993997246,10cyclictest181rcu_preempt20:09:120
31993997233,7cyclictest181rcu_preempt23:01:500
31993997233,7cyclictest181rcu_preempt23:01:500
3199399722,36cyclictest2107-21kernelversion19:04:110
32000997113,29cyclictest19626-21/usr/sbin/munin21:19:082
31997997156,9cyclictest1121-21kworker/u17:0-rpciod23:49:071
31997997140,10cyclictest0-21swapper/119:24:101
31997997140,10cyclictest0-21swapper/119:24:101
31997997124,13cyclictest181rcu_preempt20:14:191
31993997133,8cyclictest181rcu_preempt19:28:460
31993997133,8cyclictest181rcu_preempt19:28:460
32000997013,25cyclictest0-21swapper/219:48:322
31993997036,8cyclictest181rcu_preempt20:33:450
31993997036,8cyclictest181rcu_preempt20:33:450
31993997031,8cyclictest181rcu_preempt20:23:460
32000996912,33cyclictest0-21swapper/223:52:252
31993996936,8cyclictest181rcu_preempt18:58:480
31993996936,8cyclictest181rcu_preempt18:58:480
32000996810,28cyclictest0-21swapper/220:30:202
32000996810,28cyclictest0-21swapper/220:30:202
31993996833,8cyclictest181rcu_preempt19:09:250
3200799622,48cyclictest21371-21taskset20:04:023
3200799622,48cyclictest21371-21taskset20:04:023
3200799599,34cyclictest0-21swapper/322:09:043
3200799553,40cyclictest23038-21chrt00:08:213
3200799553,35cyclictest25392-21chrt22:22:393
32007995512,26cyclictest0-21swapper/300:14:003
32007995512,26cyclictest0-21swapper/300:14:003
3200799542,40cyclictest5923-21taskset21:49:193
3200799542,40cyclictest5923-21taskset21:49:193
3187925411,26sleep30-21swapper/318:58:463
3187925411,26sleep30-21swapper/318:58:463
3200799533,36cyclictest8942-21chrt21:01:473
3200799523,37cyclictest1793-21chrt00:25:523
32007995210,31cyclictest0-21swapper/321:54:313
32007995210,31cyclictest0-21swapper/321:54:313
3200799513,35cyclictest27005-21chrt22:24:173
3200799513,31cyclictest27893-21taskset20:23:593
3200799512,35cyclictest31315-21chrt21:38:543
3200799512,35cyclictest31315-21chrt21:38:543
3200799503,37cyclictest16685-21chrt21:14:083
3200799503,37cyclictest16685-21chrt21:14:083
3200799503,35cyclictest29936-21chrt23:24:163
3200799503,35cyclictest15891-21chrt21:13:433
3200799503,34cyclictest9210-21chrt22:49:223
3200799503,34cyclictest13407-21taskset19:39:073
3200799503,31cyclictest14916-21chrt19:44:063
3200799502,33cyclictest23995-21chrt21:26:293
3200799502,33cyclictest14615-21chrt23:53:543
3200799502,33cyclictest14615-21chrt23:53:543
3200799502,31cyclictest18037-21chrt19:54:023
3200799502,31cyclictest18037-21chrt19:54:023
3200799493,34cyclictest25092-21taskset23:16:093
3200799493,33cyclictest27146-21chrt23:19:173
3200799493,33cyclictest25987-21chrt20:18:453
3200799492,36cyclictest917-21kworker/3:1+events21:31:333
3200799484,32cyclictest30584-21chrt21:37:323
32007994832,9cyclictest451rcuc/322:59:193
32007994832,9cyclictest451rcuc/322:59:193
3200799483,28cyclictest14085-21chrt22:03:593
3200799483,24cyclictest21553-21chrt21:23:003
3200799474,33cyclictest11783-21chrt22:54:043
3200799473,35cyclictest11817-21chrt23:49:003
3200799473,32cyclictest17601-21taskset23:58:573
3200799472,32cyclictest8484-21taskset23:43:453
3200799472,30cyclictest5014-21chrt19:14:003
3200799464,32cyclictest6516-21chrt22:44:193
3200799464,32cyclictest6516-21chrt22:44:193
3200799463,33cyclictest1034-21chrt19:03:433
3200799463,31cyclictest4728-21chrt19:13:433
3200799463,31cyclictest2011-21chrt23:32:303
3200799462,30cyclictest25577-21chrt00:12:063
3200799454,29cyclictest32423-21chrt00:23:453
3200799453,34cyclictest4839-21chrt22:42:373
3200799453,32cyclictest31815-21chrt20:34:123
3200799452,34cyclictest20817-21chrt23:09:083
3200799452,30cyclictest29317-21chrt22:29:043
3200799452,29cyclictest24327-21chrt20:13:433
3200799452,29cyclictest24327-21chrt20:13:433
3200799444,29cyclictest3046-21taskset19:08:173
3200799442,31cyclictest10233-21chrt21:04:023
3200799442,31cyclictest10233-21chrt21:04:023
3200799433,30cyclictest2580-21chrt21:44:173
3200799433,29cyclictest5100-21chrt23:37:533
3200799433,29cyclictest5100-21chrt23:37:533
3200799432,31cyclictest19979-21taskset23:08:313
32007994226,9cyclictest0-21swapper/323:48:433
3200799422,31cyclictest4553-21taskset20:49:193
3200799422,26cyclictest11421-21taskset19:33:493
3200799413,29cyclictest32656-21chrt22:34:143
3200799413,29cyclictest32656-21chrt22:34:143
3200799413,29cyclictest16451-21chrt19:49:043
3200799413,29cyclictest16451-21chrt19:49:043
3200799413,28cyclictest2593-21taskset20:44:133
3200799413,27cyclictest8211-21chrt19:23:563
3200799413,27cyclictest8211-21chrt19:23:563
3200799413,27cyclictest19921-21chrt19:59:083
3200799412,31cyclictest12483-21chrt22:00:223
3200799412,27cyclictest29662-21taskset20:28:503
3200799412,27cyclictest29662-21taskset20:28:503
3200799403,26cyclictest5841-21chrt20:54:133
3200799403,26cyclictest22966-21taskset22:18:463
3200799403,25cyclictest9839-21taskset19:28:543
3200799393,26cyclictest7772-21chrt19:21:463
3200799393,26cyclictest7772-21chrt19:21:463
3200799383,26cyclictest27304-21taskset20:20:393
3200799382,26cyclictest1580-21taskset20:41:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional