You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-15 - 22:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Tue Jul 15, 2025 12:50:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3186728873,10sleep50-21swapper/507:09:475
3184428349,30sleep20-21swapper/207:09:292
3162928249,29sleep30-21swapper/307:06:243
3176328166,10sleep60-21swapper/607:08:216
3174327844,29sleep40-21swapper/407:08:024
3171027743,29sleep70-21swapper/707:07:347
2928027642,30sleep10-21swapper/107:05:021
2927927439,30sleep00-21swapper/007:05:010
102692590,0sleep10-21swapper/111:30:191
303242500,0sleep70-21swapper/711:55:497
24262470,0sleep30-21swapper/311:26:373
274212450,0sleep40-21swapper/411:55:134
249642420,0sleep10-21swapper/111:06:091
32520992323,0cyclictest69450irq/124-eth012:25:214
32523992218,4cyclictest630-21avahi-daemon10:29:357
243982210,0sleep40-21swapper/410:35:144
32519992010,7cyclictest0-21swapper/311:49:293
32518992015,3cyclictest734-21gmain11:43:532
32523991916,3cyclictest19802-21nscd11:02:477
2416821910,0sleep10-21swapper/111:37:251
576221810,0sleep60-21swapper/610:10:196
32523991810,4cyclictest0-21swapper/712:21:337
32522991818,0cyclictest0-21swapper/612:21:396
32521991812,6cyclictest0-21swapper/511:41:325
32516991810,8cyclictest0-21swapper/011:35:100
2001821810,5sleep40-21swapper/410:17:224
58382170,0sleep00-21swapper/011:28:500
32523991717,0cyclictest8033-21strings09:40:167
32523991710,1cyclictest0-21swapper/712:37:437
32522991717,0cyclictest12772-21ssh11:15:516
32522991716,1cyclictest27529-21ls10:05:176
32520991715,1cyclictest31598-21expr08:00:134
32520991715,1cyclictest303412sleep411:25:104
32520991715,1cyclictest24916-21sh11:53:584
32519991710,0cyclictest0-21swapper/312:20:173
32519991710,0cyclictest0-21swapper/310:07:313
32518991712,4cyclictest662-21ptp4l11:50:392
32517991710,0cyclictest0-21swapper/112:00:491
32517991710,0cyclictest0-21swapper/110:41:351
32516991716,1cyclictest27974-21ssh09:19:270
32516991716,1cyclictest101ktimersoftd/012:07:540
32516991710,3cyclictest0-21swapper/012:21:300
32516991710,3cyclictest0-21swapper/012:15:010
32523991615,1cyclictest24123-21ssh10:19:587
32523991610,4cyclictest0-21swapper/709:00:147
32523991610,0cyclictest0-21swapper/711:25:517
32523991610,0cyclictest0-21swapper/710:35:217
32522991615,1cyclictest641ktimersoftd/612:29:056
32522991610,2cyclictest0-21swapper/611:51:466
32522991610,0cyclictest0-21swapper/612:38:346
32522991610,0cyclictest0-21swapper/612:11:186
32522991610,0cyclictest0-21swapper/611:28:326
32522991610,0cyclictest0-21swapper/610:54:066
32521991614,0cyclictest9144-1kworker/u17:010:14:205
32521991613,3cyclictest5009-21sh11:28:185
32521991612,2cyclictest630-21avahi-daemon12:15:285
32521991610,0cyclictest0-21swapper/510:32:135
32520991615,1cyclictest16468-21ssh09:13:484
32520991614,1cyclictest641-21nscd12:10:324
32520991610,2cyclictest0-21swapper/410:24:144
32520991610,0cyclictest0-21swapper/412:21:504
32520991610,0cyclictest0-21swapper/412:16:444
32520991610,0cyclictest0-21swapper/411:04:234
32519991615,1cyclictest13097-21interrupts12:35:143
32519991610,2cyclictest0-21swapper/308:40:143
32519991610,0cyclictest0-21swapper/312:25:203
32519991610,0cyclictest0-21swapper/311:37:083
32518991610,5cyclictest1399-21timerwakeupswit11:10:212
32518991610,4cyclictest0-21swapper/212:02:242
32517991616,0cyclictest13903-21strings10:45:191
32517991614,2cyclictest662-21ptp4l11:47:291
32517991613,3cyclictest734-21gmain09:44:301
32517991610,0cyclictest0-21swapper/112:35:081
32517991610,0cyclictest0-21swapper/109:25:211
3251699166,9cyclictest632-21irqbalance11:18:300
32516991614,2cyclictest29729-21tr09:20:110
32516991610,5cyclictest0-21swapper/012:04:100
32516991610,3cyclictest0-21swapper/012:12:360
32516991610,2cyclictest0-21swapper/010:44:080
32523991512,3cyclictest10225-21sh11:15:137
32523991511,1cyclictest12826-21latency_hist12:20:007
32523991510,5cyclictest0-21swapper/712:06:337
32523991510,5cyclictest0-21swapper/711:44:037
32523991510,4cyclictest0-21swapper/708:20:177
32523991510,1cyclictest0-21swapper/712:29:167
32523991510,0cyclictest0-21swapper/711:47:327
32523991510,0cyclictest0-21swapper/711:33:317
32523991510,0cyclictest0-21swapper/711:21:207
32523991510,0cyclictest0-21swapper/710:31:437
32523991510,0cyclictest0-21swapper/710:04:037
32522991513,1cyclictest25624-21rm10:35:186
32522991510,5cyclictest0-21swapper/612:30:046
32522991510,0cyclictest0-21swapper/612:17:106
32522991510,0cyclictest0-21swapper/611:44:096
32522991510,0cyclictest0-21swapper/609:56:096
32522991510,0cyclictest0-21swapper/609:34:036
32521991513,1cyclictest7921-21diskmemload12:36:165
32521991513,1cyclictest7921-21diskmemload12:05:235
32521991513,1cyclictest7921-21diskmemload10:52:285
32521991513,1cyclictest7921-21diskmemload10:28:485
32521991512,2cyclictest14757-21nscd10:48:375
32521991510,0cyclictest0-21swapper/512:31:545
32521991510,0cyclictest0-21swapper/511:10:175
32520991515,0cyclictest10107-21ssh11:45:574
32520991514,1cyclictest8232-21ssh11:14:384
32520991514,1cyclictest4240-21sh10:56:074
32520991513,1cyclictest32189-21ssh09:36:124
32520991513,1cyclictest2366-21sed07:10:224
32520991510,5cyclictest0-21swapper/411:17:124
32520991510,0cyclictest0-21swapper/411:35:524
32520991510,0cyclictest0-21swapper/409:45:204
32519991513,2cyclictest401ktimersoftd/311:02:283
32519991513,1cyclictest3314-21/usr/sbin/munin12:15:223
32519991512,1cyclictest28127-21sh12:10:283
32519991510,5cyclictest0-21swapper/310:32:283
32519991510,3cyclictest4104-1kworker/u17:211:15:213
32519991510,0cyclictest0-21swapper/312:00:483
32519991510,0cyclictest0-21swapper/310:44:243
32519991510,0cyclictest0-21swapper/309:52:473
32518991512,3cyclictest3079-21munin-node10:49:472
32518991511,0cyclictest13278-21ssh11:48:042
32518991510,5cyclictest0-21swapper/212:00:002
32518991510,0cyclictest0-21swapper/211:21:382
32518991510,0cyclictest0-21swapper/211:05:112
32518991510,0cyclictest0-21swapper/209:51:182
32517991513,1cyclictest0-21swapper/107:55:151
32517991510,2cyclictest0-21swapper/109:18:201
32517991510,0cyclictest0-21swapper/112:34:301
32517991510,0cyclictest0-21swapper/112:21:451
32517991510,0cyclictest0-21swapper/112:10:131
32517991510,0cyclictest0-21swapper/111:50:561
32517991510,0cyclictest0-21swapper/111:16:551
32517991510,0cyclictest0-21swapper/110:05:131
32516991515,0cyclictest0-21swapper/011:10:420
32516991510,5cyclictest0-21swapper/010:45:040
32516991510,5cyclictest0-21swapper/009:44:580
32516991510,3cyclictest0-21swapper/011:57:050
32516991510,2cyclictest0-21swapper/012:30:470
32516991510,1cyclictest0-21swapper/012:38:050
32516991510,1cyclictest0-21swapper/011:42:090
32516991510,0cyclictest0-21swapper/011:54:290
32516991510,0cyclictest0-21swapper/011:00:210
32523991414,0cyclictest0-21swapper/712:01:307
32523991413,1cyclictest662-21ptp4l11:07:297
32523991410,4cyclictest0-21swapper/712:32:217
32523991410,4cyclictest0-21swapper/711:10:177
32523991410,4cyclictest0-21swapper/710:45:227
32523991410,0cyclictest0-21swapper/712:11:407
32523991410,0cyclictest0-21swapper/711:52:067
32523991410,0cyclictest0-21swapper/710:57:587
32523991410,0cyclictest0-21swapper/710:54:387
32523991410,0cyclictest0-21swapper/710:40:537
32523991410,0cyclictest0-21swapper/710:21:517
32523991410,0cyclictest0-21swapper/709:48:007
32523991410,0cyclictest0-21swapper/709:27:077
32522991414,0cyclictest0-21swapper/609:45:526
32522991414,0cyclictest0-21swapper/609:42:496
32522991414,0cyclictest0-21swapper/609:17:596
32522991412,2cyclictest5467-1kworker/u17:010:57:596
32522991410,4cyclictest0-21swapper/611:31:556
32522991410,4cyclictest0-21swapper/611:11:476
32522991410,4cyclictest0-21swapper/611:08:426
32522991410,4cyclictest0-21swapper/609:22:476
32522991410,3cyclictest0-21swapper/610:40:176
32522991410,1cyclictest0-21swapper/608:00:186
32522991410,0cyclictest0-21swapper/612:09:386
32522991410,0cyclictest0-21swapper/612:02:476
32522991410,0cyclictest0-21swapper/611:45:586
32522991410,0cyclictest0-21swapper/611:38:066
32522991410,0cyclictest0-21swapper/611:00:316
32522991410,0cyclictest0-21swapper/610:18:306
32522991410,0cyclictest0-21swapper/610:00:186
32522991410,0cyclictest0-21swapper/609:36:266
32522991410,0cyclictest0-21swapper/609:25:336
32522991410,0cyclictest0-21swapper/608:35:156
32521991414,0cyclictest0-21swapper/507:20:165
32521991413,1cyclictest7921-21diskmemload10:42:495
32521991413,1cyclictest28879-21ssh12:10:585
32521991413,1cyclictest27622-21ssh11:23:485
32521991413,1cyclictest19483-21ssh09:45:265
32521991413,1cyclictest10182-21ssh09:56:545
32521991412,1cyclictest7921-21diskmemload11:34:225
32521991410,4cyclictest0-21swapper/510:03:075
32521991410,0cyclictest0-21swapper/512:20:575
32521991410,0cyclictest0-21swapper/511:35:055
32521991410,0cyclictest0-21swapper/511:07:585
32521991410,0cyclictest0-21swapper/508:45:135
32520991414,0cyclictest0-21swapper/412:04:504
32520991413,1cyclictest792-21ssh10:08:224
32520991413,1cyclictest32304-21ssh09:20:314
32520991413,1cyclictest19010-21ssh10:48:064
32520991412,1cyclictest8451-21sh10:27:144
32520991412,1cyclictest17596-21ls12:05:214
32520991412,1cyclictest12896-21rm10:14:274
32520991410,4cyclictest0-21swapper/409:50:234
32520991410,3cyclictest0-21swapper/411:21:204
32520991410,3cyclictest0-21swapper/410:01:194
32520991410,1cyclictest0-21swapper/410:50:204
32520991410,1cyclictest0-21swapper/409:33:384
32520991410,0cyclictest0-21swapper/412:33:424
32520991410,0cyclictest0-21swapper/410:41:304
32520991410,0cyclictest0-21swapper/408:19:104
32519991414,0cyclictest0-21swapper/311:22:213
32519991414,0cyclictest0-21swapper/310:56:023
32519991413,1cyclictest9802-21latency_hist07:25:003
32519991413,0cyclictest192-21md0_raid109:20:163
32519991412,1cyclictest7921-21diskmemload11:10:573
32519991412,1cyclictest7921-21diskmemload11:05:273
32519991410,4cyclictest0-21swapper/311:40:333
32519991410,4cyclictest0-21swapper/309:40:463
32519991410,3cyclictest0-21swapper/310:25:113
32519991410,2cyclictest0-21swapper/312:05:123
32519991410,2cyclictest0-21swapper/309:57:423
32519991410,2cyclictest0-21swapper/309:12:513
32519991410,0cyclictest0-21swapper/312:30:023
32519991410,0cyclictest0-21swapper/311:55:163
32519991410,0cyclictest0-21swapper/310:46:303
32519991410,0cyclictest0-21swapper/310:24:463
32519991410,0cyclictest0-21swapper/310:16:553
32519991410,0cyclictest0-21swapper/310:13:193
32519991410,0cyclictest0-21swapper/310:03:113
32519991410,0cyclictest0-21swapper/309:47:503
32519991410,0cyclictest0-21swapper/309:31:053
32519991410,0cyclictest0-21swapper/307:10:223
32518991414,0cyclictest0-21swapper/212:08:192
32518991414,0cyclictest0-21swapper/209:24:062
32518991413,1cyclictest16750irq/126-ahci[0012:20:002
32518991413,0cyclictest641-21nscd11:19:532
32518991412,1cyclictest641-21nscd11:00:172
32518991410,4cyclictest0-21swapper/212:24:252
32518991410,4cyclictest0-21swapper/210:40:312
32518991410,4cyclictest0-21swapper/210:14:182
32518991410,4cyclictest0-21swapper/209:36:122
32518991410,4cyclictest0-21swapper/209:26:052
32518991410,0cyclictest0-21swapper/212:37:432
32518991410,0cyclictest0-21swapper/212:25:522
32518991410,0cyclictest0-21swapper/212:14:032
32518991410,0cyclictest0-21swapper/211:28:382
32518991410,0cyclictest0-21swapper/210:50:462
32518991410,0cyclictest0-21swapper/210:37:212
32518991410,0cyclictest0-21swapper/210:22:582
32518991410,0cyclictest0-21swapper/210:08:022
32518991410,0cyclictest0-21swapper/209:48:192
32518991410,0cyclictest0-21swapper/209:30:172
32518991410,0cyclictest0-21swapper/209:18:302
32518991410,0cyclictest0-21swapper/209:10:192
32517991412,1cyclictest641-21nscd10:00:181
32517991410,4cyclictest0-21swapper/110:26:561
32517991410,4cyclictest0-21swapper/110:20:581
32517991410,4cyclictest0-21swapper/109:50:161
32517991410,4cyclictest0-21swapper/109:45:431
32517991410,4cyclictest0-21swapper/109:24:261
32517991410,2cyclictest0-21swapper/111:25:161
32517991410,2cyclictest0-21swapper/108:40:131
32517991410,0cyclictest0-21swapper/112:25:131
32517991410,0cyclictest0-21swapper/112:05:091
32517991410,0cyclictest0-21swapper/111:56:261
32517991410,0cyclictest0-21swapper/111:41:131
32517991410,0cyclictest0-21swapper/111:20:011
32517991410,0cyclictest0-21swapper/111:14:041
32517991410,0cyclictest0-21swapper/111:00:211
32517991410,0cyclictest0-21swapper/110:55:151
32517991410,0cyclictest0-21swapper/110:36:211
32517991410,0cyclictest0-21swapper/110:15:091
32517991410,0cyclictest0-21swapper/109:35:201
32516991414,0cyclictest19248-21ssh10:01:030
32516991414,0cyclictest0-21swapper/011:49:370
32516991414,0cyclictest0-21swapper/010:19:130
32516991413,1cyclictest8350-21kworker/u16:009:35:280
32516991410,4cyclictest0-21swapper/011:32:150
32516991410,4cyclictest0-21swapper/011:22:040
32516991410,3cyclictest0-21swapper/012:25:140
32516991410,3cyclictest0-21swapper/010:53:230
32516991410,2cyclictest0-21swapper/010:11:100
32516991410,1cyclictest111rcuc/011:05:170
32516991410,0cyclictest0-21swapper/010:58:450
32516991410,0cyclictest0-21swapper/010:36:380
32516991410,0cyclictest0-21swapper/010:25:340
32516991410,0cyclictest0-21swapper/009:53:530
316921410,3sleep50-21swapper/511:59:155
53302130,0sleep20-21swapper/210:56:512
32523991313,0cyclictest0-21swapper/711:35:177
32523991310,3cyclictest0-21swapper/709:20:127
32523991310,3cyclictest0-21swapper/709:16:187
32523991310,3cyclictest0-21swapper/708:06:477
32523991310,3cyclictest0-21swapper/708:00:007
32523991310,1cyclictest0-21swapper/708:15:207
32523991310,0cyclictest0-21swapper/710:05:157
32523991310,0cyclictest0-21swapper/709:56:177
32523991310,0cyclictest0-21swapper/709:51:297
32523991310,0cyclictest0-21swapper/709:39:207
32523991310,0cyclictest0-21swapper/709:30:127
32523991310,0cyclictest0-21swapper/709:10:027
32523991310,0cyclictest0-21swapper/708:40:187
32523991310,0cyclictest0-21swapper/708:27:457
32523991310,0cyclictest0-21swapper/708:10:167
32523991310,0cyclictest0-21swapper/707:40:207
32522991311,1cyclictest662-21ptp4l07:41:276
32522991310,3cyclictest0-21swapper/609:10:206
32522991310,3cyclictest0-21swapper/608:28:176
32522991310,0cyclictest0-21swapper/611:55:446
32522991310,0cyclictest0-21swapper/611:20:346
32522991310,0cyclictest0-21swapper/610:47:056
32522991310,0cyclictest0-21swapper/610:30:136
32522991310,0cyclictest0-21swapper/610:25:066
32522991310,0cyclictest0-21swapper/610:20:076
32522991310,0cyclictest0-21swapper/609:51:316
32522991310,0cyclictest0-21swapper/609:04:306
32522991310,0cyclictest0-21swapper/607:20:216
32521991313,0cyclictest30095-21ls07:55:235
32521991312,1cyclictest30945-21ssh09:20:155
32521991312,1cyclictest12374-21ssh11:47:275
32521991311,1cyclictest650-21nscd09:15:215
32521991311,1cyclictest647-21nscd11:17:455
32521991311,1cyclictest647-21nscd09:50:535
32521991311,1cyclictest2118-21ssh09:37:405
32521991311,1cyclictest13291-21rm09:43:095
32521991311,0cyclictest121rcu_preempt10:24:085
32521991310,3cyclictest0-21swapper/511:00:165
32521991310,3cyclictest0-21swapper/510:10:005
32521991310,0cyclictest0-21swapper/512:26:505
32521991310,0cyclictest0-21swapper/512:04:355
32521991310,0cyclictest0-21swapper/510:19:425
32521991310,0cyclictest0-21swapper/507:50:175
32520991313,0cyclictest7544-21ls09:55:224
32520991313,0cyclictest69450irq/124-eth008:45:094
32520991312,1cyclictest69450irq/124-eth011:05:204
32520991312,1cyclictest69450irq/124-eth009:29:164
32520991312,1cyclictest32221-21ssh11:40:564
32520991311,1cyclictest5612-21head07:15:224
32520991311,1cyclictest5497-21nscd12:35:064
32520991311,1cyclictest3799-21needreboot09:00:204
32520991311,1cyclictest11373-21rm11:30:454
32520991310,3cyclictest0-21swapper/410:30:424
32520991310,3cyclictest0-21swapper/407:45:164
32520991310,0cyclictest0-21swapper/409:17:344
32519991313,0cyclictest13510-1kworker/u17:211:51:013
32519991312,1cyclictest19913-21tune2fs07:40:133
32519991312,1cyclictest1585-21lldpd10:50:083
32519991311,1cyclictest641-21nscd11:30:153
32519991311,1cyclictest28082-21taskset08:46:083
32519991310,3cyclictest0-21swapper/307:30:133
32519991310,1cyclictest0-21swapper/310:35:033
32519991310,1cyclictest0-21swapper/309:15:193
32519991310,0cyclictest0-21swapper/309:35:583
32519991310,0cyclictest0-21swapper/309:27:353
32519991310,0cyclictest0-21swapper/308:05:173
32518991311,1cyclictest641-21nscd11:35:092
32518991311,1cyclictest641-21nscd10:03:182
32518991311,1cyclictest14013-21df_abs10:30:122
32518991310,3cyclictest0-21swapper/209:05:162
32518991310,1cyclictest0-21swapper/212:30:132
32518991310,0cyclictest0-21swapper/211:30:452
32518991310,0cyclictest0-21swapper/210:27:322
32518991310,0cyclictest0-21swapper/210:16:022
32518991310,0cyclictest0-21swapper/209:55:162
32518991310,0cyclictest0-21swapper/209:42:342
32518991310,0cyclictest0-21swapper/209:03:382
32518991310,0cyclictest0-21swapper/208:39:502
32518991310,0cyclictest0-21swapper/207:25:182
32517991313,0cyclictest17832-21ls07:35:201
32517991311,1cyclictest7921-21diskmemload10:30:261
32517991311,1cyclictest641-21nscd10:50:221
32517991310,3cyclictest0-21swapper/112:15:171
32517991310,3cyclictest0-21swapper/108:26:221
32517991310,3cyclictest0-21swapper/108:04:321
32517991310,3cyclictest0-21swapper/107:30:141
32517991310,3cyclictest0-21swapper/107:13:031
32517991310,0cyclictest0-21swapper/110:10:211
32517991310,0cyclictest0-21swapper/109:55:181
32517991310,0cyclictest0-21swapper/109:32:481
32517991310,0cyclictest0-21swapper/109:10:211
32517991310,0cyclictest0-21swapper/109:00:481
32517991310,0cyclictest0-21swapper/108:50:121
32517991310,0cyclictest0-21swapper/108:45:171
32517991310,0cyclictest0-21swapper/107:50:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional