You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-18 - 15:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Fri Apr 18, 2025 12:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
413540826352,7sleep40-21swapper/415:34:054
413540225318,30sleep70-21swapper/715:34:007
413558824838,7sleep50-21swapper/515:36:395
413568124535,7sleep60-21swapper/615:38:006
4136004993330,3cyclictest70835-21kworker/u16:2+flush-8:017:43:316
4136000993329,3cyclictest4179935-21kworker/u16:2+flush-8:016:08:225
4136000993230,2cyclictest353190-21kworker/u16:4+events_unbound19:44:565
4136004993129,2cyclictest4146224-21kworker/u16:4+events_unbound16:28:206
4136004993129,2cyclictest4146224-21kworker/u16:4+events_unbound16:28:206
4136004993126,4cyclictest405853-21packagekitd19:44:276
4136000993128,2cyclictest363536-21kworker/u16:0+events_unbound19:20:155
4136000993128,2cyclictest132093-21kworker/u16:0+events_unbound17:54:355
4136004993027,2cyclictest385158-21kworker/u16:1+flush-8:020:18:056
4136000992927,2cyclictest132093-21kworker/u16:0+events_unbound17:50:155
4136000992926,2cyclictest449324-21kworker/u16:4+events_unbound20:28:145
4136000992926,2cyclictest132093-21kworker/u16:0+events_unbound17:18:205
4136004992826,2cyclictest4192005-21kworker/u16:3+events_unbound16:03:316
4136000992826,2cyclictest4130863-21kworker/u16:3+events_unbound15:41:075
4136000992826,2cyclictest4057937-21kworker/u16:1+events_unbound15:45:075
413600899270,25cyclictest438298-21latency_hist20:08:057
4136000992725,2cyclictest70835-21kworker/u16:2+flush-8:018:26:275
4136000992725,2cyclictest385158-21kworker/u16:1+events_unbound20:14:075
4136000992725,2cyclictest374232-21kworker/u16:3+events_unbound20:11:065
4136000992624,2cyclictest49112-21kworker/u16:1+events_unbound16:34:595
4136000992624,2cyclictest225643-21kworker/u16:1+events_unbound18:30:075
4136000992623,2cyclictest416864-21kworker/u16:0+events_unbound20:23:075
4136000992623,2cyclictest416864-21kworker/u16:0+events_unbound20:23:075
4136004992523,2cyclictest225643-21kworker/u16:1+flush-8:018:31:356
4136000992523,2cyclictest70835-21kworker/u16:2+events_unbound18:46:075
4136000992523,2cyclictest70835-21kworker/u16:2+events_unbound18:41:355
4136000992523,2cyclictest70835-21kworker/u16:2+events_unbound17:15:405
4136000992523,2cyclictest70835-21kworker/u16:2+events_unbound17:15:405
4136000992523,2cyclictest70835-21kworker/u16:2+events_unbound16:47:165
4136000992523,2cyclictest449324-21kworker/u16:4+events_unbound20:42:515
4136000992523,2cyclictest449324-21kworker/u16:4+events_unbound20:33:595
4136000992523,2cyclictest4179935-21kworker/u16:2+events_unbound16:04:205
4136000992523,2cyclictest298903-21kworker/u16:0+events_unbound19:03:155
4136000992523,2cyclictest298903-21kworker/u16:0+events_unbound18:49:545
4136000992523,2cyclictest298903-21kworker/u16:0+events_unbound18:49:545
4136000992523,2cyclictest256675-21kworker/u16:3+flush-8:019:10:395
4136000992523,2cyclictest225643-21kworker/u16:1+events_unbound18:09:285
4136000992523,2cyclictest19653-21kworker/u16:3+flush-8:016:20:205
4136000992523,2cyclictest19653-21kworker/u16:3+events_unbound16:27:085
4136000992523,2cyclictest184823-21kworker/u16:1+flush-8:017:36:075
4136000992523,2cyclictest133277-21kworker/u16:3+flush-8:017:46:525
4136000992522,2cyclictest374232-21kworker/u16:3+flush-8:019:56:245
4136004992422,2cyclictest385158-21kworker/u16:1+flush-8:020:28:366
4136000992422,2cyclictest81159-21kworker/u16:0+flush-8:016:51:195
4136000992422,2cyclictest70835-21kworker/u16:2+events_unbound19:01:085
4136000992422,2cyclictest497174-21kworker/u16:2+events_unbound20:53:465
4136000992422,2cyclictest49112-21kworker/u16:1+events_unbound17:11:205
4136000992422,2cyclictest49112-21kworker/u16:1+events_unbound17:01:415
4136000992422,2cyclictest4179935-21kworker/u16:2+events_unbound16:15:525
4136000992422,2cyclictest385158-21kworker/u16:1+flush-8:020:00:195
4136000992422,2cyclictest374013-21kworker/u16:2+events_unbound19:40:445
4136000992422,2cyclictest374013-21kworker/u16:2+events_unbound19:30:195
4136000992422,2cyclictest225643-21kworker/u16:1+events_unbound19:24:235
4136000992422,2cyclictest225643-21kworker/u16:1+events_unbound18:33:195
4136000992422,2cyclictest225643-21kworker/u16:1+events_unbound18:19:205
4136004992321,2cyclictest479527-21kworker/u16:3+events_unbound20:38:066
4136004992321,2cyclictest416864-21kworker/u16:0+flush-8:020:03:356
4136004992321,2cyclictest132093-21kworker/u16:0+flush-8:017:48:396
4136000992321,2cyclictest70835-21kworker/u16:2+flush-8:018:01:125
4136000992321,2cyclictest70835-21kworker/u16:2+events_unbound17:25:205
4136000992321,2cyclictest70835-21kworker/u16:2+events_unbound17:07:205
4136000992321,2cyclictest49112-21kworker/u16:1+flush-8:016:57:435
4136000992321,2cyclictest449324-21kworker/u16:4+flush-8:020:58:155
4136000992321,2cyclictest4146224-21kworker/u16:4+events_unbound16:40:125
4136000992321,2cyclictest374232-21kworker/u16:3+events_unbound19:49:425
4136000992321,2cyclictest256675-21kworker/u16:3+flush-8:019:13:445
4136000992321,2cyclictest132093-21kworker/u16:0+flush-8:018:05:075
4136000992320,2cyclictest298903-21kworker/u16:0+events_unbound18:56:325
4136000992320,2cyclictest256675-21kworker/u16:3+events_unbound18:13:275
4136008992217,3cyclictest353548-21idleruntime-cro19:18:057
4136004992220,2cyclictest4146224-21kworker/u16:4+flush-8:016:33:346
4136000992220,2cyclictest479527-21kworker/u16:3+flush-8:020:45:265
4136000992220,2cyclictest416864-21kworker/u16:0+events_unbound20:21:525
4136000992220,2cyclictest416864-21kworker/u16:0+events_unbound20:04:075
4136000992220,2cyclictest4146224-21kworker/u16:4+events_unbound16:31:245
4136000992220,2cyclictest4146224-21kworker/u16:4+events_unbound16:31:245
4136000992220,2cyclictest4146224-21kworker/u16:4+events_unbound15:49:195
4136000992220,2cyclictest4057937-21kworker/u16:1+events_unbound16:02:195
4136000992220,2cyclictest385158-21kworker/u16:1+events_unbound21:04:235
4136000992220,2cyclictest132093-21kworker/u16:0+events_unbound17:32:205
413600899210,19cyclictest491891-21latency_hist20:43:057
413600899210,19cyclictest416627-21latency_hist19:53:057
4136004992118,2cyclictest479527-21kworker/u16:3+flush-8:020:53:156
4136000992119,2cyclictest385158-21kworker/u16:1+events_unbound19:37:115
413600899200,18cyclictest948-21NetworkManager19:43:307
413600899200,18cyclictest100994-21latency_hist16:53:067
4136004992018,2cyclictest4191473-21kworker/u16:0+flush-8:016:13:316
4136004992018,2cyclictest405911-21kworker/u16:5+flush-8:019:50:356
4136004992018,2cyclictest256675-21kworker/u16:3+events_unbound18:59:076
4136000992018,2cyclictest479527-21kworker/u16:3+flush-8:020:50:195
4136000992018,2cyclictest4057937-21kworker/u16:1+events_unbound15:55:485
413600899190,17cyclictest0-21swapper/717:13:347
413600899190,17cyclictest0-21swapper/717:13:347
4136004991917,2cyclictest4179935-21kworker/u16:2+flush-8:015:55:156
4136004991917,2cyclictest4146224-21kworker/u16:4+events_unbound16:56:156
4136004991917,2cyclictest4118042-21kworker/u16:0+flush-8:015:43:276
4136004991917,2cyclictest4057937-21kworker/u16:1+flush-8:015:49:066
4136004991917,2cyclictest363536-21kworker/u16:0+flush-8:019:21:356
4136004991917,2cyclictest184823-21kworker/u16:1+flush-8:017:33:236
4136004991915,3cyclictest132093-21kworker/u16:0+events_unbound17:08:306
413600499183,14cyclictest0-21swapper/620:08:246
4136004991816,2cyclictest70835-21kworker/u16:2+flush-8:018:28:016
4136004991816,2cyclictest497174-21kworker/u16:2+flush-8:020:48:266
4136004991816,2cyclictest405911-21kworker/u16:5+events_unbound20:01:156
4136004991816,2cyclictest298903-21kworker/u16:0+events_unbound19:08:356
4136004991816,2cyclictest132093-21kworker/u16:0+flush-8:018:03:216
4136004991815,3cyclictest70835-21kworker/u16:2+flush-8:016:38:346
4136004991715,2cyclictest70835-21kworker/u16:2+events_unbound18:39:076
4136004991715,2cyclictest497174-21kworker/u16:2+events_unbound20:44:156
4136004991715,2cyclictest4146224-21kworker/u16:4+flush-8:016:28:016
4136004991715,2cyclictest225643-21kworker/u16:1+flush-8:017:58:186
4136004991715,2cyclictest132093-21kworker/u16:0+flush-8:017:31:156
4136004991714,2cyclictest298903-21kworker/u16:0+events_unbound18:48:226
4136004991714,2cyclictest298903-21kworker/u16:0+events_unbound18:48:226
4136008991615,0cyclictest0-21swapper/717:18:247
4136004991614,2cyclictest465365-21kworker/u16:2+flush-8:020:23:366
4136004991614,2cyclictest465365-21kworker/u16:2+flush-8:020:23:366
4136004991614,2cyclictest385158-21kworker/u16:1+events_unbound19:54:356
4136004991614,2cyclictest374232-21kworker/u16:3+events_unbound19:41:356
4136004991614,2cyclictest298903-21kworker/u16:0+events_unbound18:57:356
4136004991613,2cyclictest133277-21kworker/u16:3+events_unbound17:13:236
4136004991613,2cyclictest133277-21kworker/u16:3+events_unbound17:13:236
4135996991615,1cyclictest0-21swapper/419:18:244
413599699160,15cyclictest127464-21kthreadcore17:03:254
413600899151,13cyclictest470220-21turbostat20:28:057
413600899151,13cyclictest470220-21turbostat20:28:057
4136004991513,2cyclictest70835-21kworker/u16:2+events_unbound18:43:236
4136004991513,2cyclictest363536-21kworker/u16:0+events_unbound19:27:356
4136008991412,1cyclictest331793-21turbostat19:03:057
413600899140,12cyclictest417718-21latency_hist19:58:057
4136004991412,2cyclictest70835-21kworker/u16:2+flush-8:019:08:016
4136004991412,2cyclictest70835-21kworker/u16:2+flush-8:018:17:356
4136004991412,2cyclictest4146224-21kworker/u16:4+events_unbound15:58:286
4136004991412,2cyclictest133277-21kworker/u16:3+flush-8:017:18:316
4135996991413,1cyclictest0-21swapper/415:43:114
413600899132,10cyclictest142365-21cat17:13:057
413600899132,10cyclictest1313-21snmpd18:23:217
4136008991311,1cyclictest320396-21turbostat18:53:057
4136008991311,1cyclictest320396-21turbostat18:53:057
413600899130,2cyclictest236616-21/usr/sbin/munin18:03:337
413600899130,11cyclictest70102-21turbostat16:38:057
4136004991311,2cyclictest257699-21kworker/u16:4+flush-8:018:22:476
4136004991310,2cyclictest81159-21kworker/u16:0+flush-8:016:48:226
4136004991310,2cyclictest4146224-21kworker/u16:4+events_unbound17:03:096
413599699132,3cyclictest266154-21kthreadcore18:18:234
4135996991312,1cyclictest0-21swapper/420:59:164
4135996991310,2cyclictest30557-21irqcore16:18:274
413599699130,1cyclictest0-21swapper/416:34:164
413599699130,1cyclictest0-21swapper/416:26:514
413599699130,13cyclictest65-21ksoftirqd/415:50:354
413599699130,13cyclictest0-21swapper/417:35:164
413600899123,7cyclictest27203-21xdg-desktop-por18:35:537
4136008991211,0cyclictest0-21swapper/716:38:437
4136008991210,1cyclictest0-21swapper/718:43:287
413600899120,4cyclictest4181708-21/usr/sbin/munin15:58:367
413600899120,2cyclictest27818-21ntpq16:13:317
413600899120,11cyclictest96-21ksoftirqd/716:23:227
413600899120,11cyclictest448558-21latency_hist20:13:057
413600899120,11cyclictest121553-21turbostat17:03:057
413600899120,10cyclictest1313-21snmpd21:06:207
413600899120,10cyclictest1313-21snmpd20:56:077
413600899120,10cyclictest1313-21snmpd20:52:377
413600899120,10cyclictest1313-21snmpd17:28:357
413600899120,10cyclictest1313-21snmpd17:08:017
413600499129,2cyclictest4146224-21kworker/u16:4+events_unbound16:58:166
4136004991210,2cyclictest70835-21kworker/u16:2+events_unbound17:26:366
413600499120,1cyclictest0-21swapper/620:33:306
413600499120,1cyclictest0-21swapper/615:41:336
413600499120,12cyclictest0-21swapper/616:44:456
413599699129,2cyclictest0-21swapper/420:28:224
413599699122,4cyclictest21614-21grep16:13:244
4135996991211,1cyclictest65-21ksoftirqd/418:23:494
4135996991211,1cyclictest0-21swapper/420:44:364
4135996991211,1cyclictest0-21swapper/420:39:434
4135996991211,1cyclictest0-21swapper/418:33:364
4135996991211,1cyclictest0-21swapper/416:51:174
4135996991211,1cyclictest0-21swapper/416:39:094
413599699120,1cyclictest0-21swapper/419:25:374
413599699120,12cyclictest65-21ksoftirqd/420:10:144
413599699120,12cyclictest65-21ksoftirqd/415:39:524
413599699120,12cyclictest0-21swapper/420:15:074
413599699120,12cyclictest0-21swapper/418:07:264
413600899115,5cyclictest27763-21gvfs-afc-volume18:38:267
413600899111,9cyclictest4136301-21cyclictest19:40:387
413600899111,9cyclictest0-21swapper/718:28:257
413600899111,2cyclictest204635-21/usr/sbin/munin17:43:227
4136008991111,0cyclictest0-21swapper/719:37:237
4136008991110,0cyclictest0-21swapper/720:18:137
4136008991110,0cyclictest0-21swapper/718:55:367
4136008991110,0cyclictest0-21swapper/717:27:007
413600899110,4cyclictest224262-21sshd-session17:49:337
413600899110,2cyclictest4175013-21kthreadcore15:53:287
413600899110,2cyclictest365805-21kthreadcore19:23:257
413600899110,2cyclictest353092-21sed19:13:067
413600899110,1cyclictest0-21swapper/719:58:307
413600899110,1cyclictest0-21swapper/716:53:207
413600899110,11cyclictest96-21ksoftirqd/716:28:297
413600899110,11cyclictest96-21ksoftirqd/716:28:297
413600899110,10cyclictest479563-21ntpq20:28:327
413600899110,10cyclictest4150874-21/usr/sbin/munin15:43:317
413600899110,10cyclictest1313-21snmpd19:07:407
413600899110,10cyclictest1313-21snmpd18:19:037
413600899110,10cyclictest1313-21snmpd18:17:307
413600899110,10cyclictest1313-21snmpd17:35:237
413600899110,10cyclictest1313-21snmpd16:08:457
413600899110,10cyclictest1313-21snmpd15:50:357
4136004991110,1cyclictest0-21swapper/621:03:376
4136004991110,1cyclictest0-21swapper/616:21:546
413600499110,11cyclictest0-21swapper/620:21:126
413600499110,11cyclictest0-21swapper/619:31:526
413600499110,11cyclictest0-21swapper/618:36:326
413600499110,11cyclictest0-21swapper/616:10:236
413599699112,2cyclictest112152-21cpuspeed_turbos16:58:224
413599699111,9cyclictest0-21swapper/417:18:244
413599699111,6cyclictest3792081kthreadcore19:28:274
413599699111,5cyclictest139151kthreadcore16:08:274
413599699111,3cyclictest291985-21kthreadcore18:38:194
4135996991110,1cyclictest0-21swapper/419:16:384
4135996991110,1cyclictest0-21swapper/419:11:454
4135996991110,1cyclictest0-21swapper/418:47:194
4135996991110,1cyclictest0-21swapper/418:10:444
4135996991110,1cyclictest0-21swapper/417:08:194
4135996991110,1cyclictest0-21swapper/416:44:494
4135996991110,1cyclictest0-21swapper/416:29:224
4135996991110,1cyclictest0-21swapper/416:29:224
413599699110,11cyclictest0-21swapper/420:54:224
413599699110,11cyclictest0-21swapper/420:02:494
413599699110,11cyclictest0-21swapper/419:07:394
413599699110,11cyclictest0-21swapper/418:28:434
413599699110,11cyclictest0-21swapper/418:15:374
413599699110,11cyclictest0-21swapper/416:03:504
413599699110,11cyclictest0-21swapper/415:58:004
413600899108,1cyclictest4146212-21awk15:43:057
413600899102,0cyclictest0-21swapper/716:18:307
413600899101,8cyclictest1313-21snmpd19:28:327
413600899100,9cyclictest1313-21snmpd20:14:097
413600899100,9cyclictest1313-21snmpd17:55:207
413600899100,9cyclictest1313-21snmpd16:05:337
413600899100,9cyclictest0-21swapper/716:43:317
413600899100,8cyclictest27975-21passimd20:43:237
413600899100,2cyclictest948-21NetworkManager20:35:537
413600899100,1cyclictest0-21swapper/719:18:237
413600899100,1cyclictest0-21swapper/717:58:187
413600499108,2cyclictest374232-21kworker/u16:3+events_unbound19:34:356
413599699109,1cyclictest0-21swapper/420:18:264
413599699109,1cyclictest0-21swapper/419:45:294
413599699109,1cyclictest0-21swapper/418:52:594
413599699109,1cyclictest0-21swapper/418:52:594
413599699109,1cyclictest0-21swapper/417:54:214
413599699109,1cyclictest0-21swapper/416:00:314
413599699107,2cyclictest0-21swapper/420:48:334
413599699102,2cyclictest386690-21cat19:38:144
413599699101,6cyclictest433287-21kthreadcore20:03:254
41360089991,0cyclictest0-21swapper/718:08:267
41360089990,7cyclictest515137-21irqcore20:58:177
41360049997,2cyclictest298903-21kworker/u16:0+events_unbound19:13:156
41360049996,2cyclictest497174-21kworker/u16:2+flush-8:020:58:106
41360049996,2cyclictest225643-21kworker/u16:1+events_unbound18:08:086
41359969998,1cyclictest0-21swapper/421:04:094
41359969998,1cyclictest0-21swapper/418:02:324
41359969998,1cyclictest0-21swapper/417:51:024
41359969998,1cyclictest0-21swapper/417:44:254
41359969998,1cyclictest0-21swapper/417:31:574
41359969998,1cyclictest0-21swapper/416:57:454
41359969997,2cyclictest0-21swapper/419:01:584
41359969996,2cyclictest0-21swapper/420:23:254
41359969996,2cyclictest0-21swapper/420:23:254
41359969991,7cyclictest146327-21kthreadcore17:13:274
41359969991,7cyclictest146327-21kthreadcore17:13:274
41360049987,1cyclictest70835-21kworker/u16:2+flush-8:017:55:036
41359969987,1cyclictest0-21swapper/420:35:374
41359969987,1cyclictest0-21swapper/419:33:554
41359969984,3cyclictest417606-21taskset19:56:374
41359969983,3cyclictest65-21ksoftirqd/419:49:554
41359969980,7cyclictest321451-21chrt18:57:484
41359969980,7cyclictest0-21swapper/417:23:224
41359969960,2cyclictest65-21ksoftirqd/417:43:074
41359969960,2cyclictest65-21ksoftirqd/417:43:074
41360089950,4cyclictest203894-21latency_hist17:43:077
41360089950,4cyclictest203894-21latency_hist17:43:077
41360009930,1cyclictest991rcuop/717:43:075
41360009930,1cyclictest991rcuop/717:43:075
41360049910,1cyclictest0-21swapper/617:43:076
41360049910,1cyclictest0-21swapper/617:43:076
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional