You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 06:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot3s.osadl.org (updated Thu Nov 13, 2025 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7659202280271,6sleep70-21swapper/719:06:517
7661142275212,53sleep10-21swapper/119:09:271
7659652259216,33sleep00-21swapper/019:07:240
7638302258206,17sleep50-21swapper/519:05:045
7659412248221,17sleep30-21swapper/319:07:023
7661372247221,17sleep60-21swapper/619:09:486
7659332246219,17sleep40-21swapper/419:06:554
7658782246221,16sleep20-21swapper/219:06:142
766449993329,3cyclictest943300-21cat00:30:176
766439993332,1cyclictest0-21swapper/421:50:204
766449993130,1cyclictest0-21swapper/622:05:206
766430993024,4cyclictest0-21swapper/223:50:162
76644999290,27cyclictest0-21swapper/620:23:066
766430992927,1cyclictest0-21swapper/222:00:222
766449992821,6cyclictest1377-21snmpd20:07:246
766449992821,6cyclictest1377-21snmpd20:07:236
76643499280,28cyclictest0-21swapper/322:00:223
76644999270,25cyclictest0-21swapper/622:23:436
76642199270,25cyclictest0-21swapper/023:46:010
76645699260,22cyclictest0-21swapper/720:51:317
76645699260,22cyclictest0-21swapper/719:10:107
766449992621,4cyclictest1377-21snmpd22:42:536
766449992621,4cyclictest1377-21snmpd22:42:536
76644999260,24cyclictest0-21swapper/622:50:086
76644399260,22cyclictest0-21swapper/519:18:025
766439992625,1cyclictest0-21swapper/419:20:174
766439992622,3cyclictest937838-21cat00:20:174
766439992617,5cyclictest0-21swapper/422:14:164
76643999260,22cyclictest0-21swapper/423:58:174
76643999260,22cyclictest0-21swapper/423:39:554
76643999260,22cyclictest0-21swapper/419:26:054
76643499260,25cyclictest803497-21expr20:15:153
76643099268,2cyclictest0-21swapper/221:35:172
76642199260,24cyclictest0-21swapper/021:02:530
766456992518,5cyclictest0-21swapper/722:59:337
766456992518,3cyclictest0-21swapper/722:10:167
76645699250,23cyclictest0-21swapper/722:42:347
76645699250,23cyclictest0-21swapper/722:42:337
76645699250,23cyclictest0-21swapper/721:44:267
76645699250,23cyclictest0-21swapper/720:49:507
76645699250,23cyclictest0-21swapper/720:14:467
76645699250,23cyclictest0-21swapper/700:01:547
76645699250,1cyclictest0-21swapper/719:56:577
766449992523,2cyclictest0-21swapper/622:48:056
766449992521,3cyclictest1377-21snmpd20:42:046
766449992518,5cyclictest0-21swapper/622:29:326
766449992518,5cyclictest0-21swapper/619:36:206
76644999251,24cyclictest0-21swapper/619:29:136
76644399250,1cyclictest0-21swapper/523:05:175
766439992522,2cyclictest1377-21snmpd21:20:374
766439992521,3cyclictest1377-21snmpd23:03:214
766439992518,5cyclictest0-21swapper/422:01:164
766439992517,6cyclictest0-21swapper/419:14:024
76643999251,22cyclictest0-21swapper/421:00:404
766434992524,1cyclictest0-21swapper/323:50:163
766434992518,5cyclictest0-21swapper/321:09:323
766434992518,3cyclictest0-21swapper/323:48:403
766434992511,14cyclictest1377-21snmpd20:25:213
76643499250,1cyclictest461irq_work/321:19:593
76643499250,1cyclictest0-21swapper/320:55:513
766430992521,3cyclictest1377-21snmpd20:58:252
766430992518,5cyclictest0-21swapper/220:54:582
766425992522,3cyclictest1377-21snmpd21:16:141
766421992521,3cyclictest1377-21snmpd22:24:310
766421992518,3cyclictest0-21swapper/020:05:300
766421992518,3cyclictest0-21swapper/020:05:300
766456992420,3cyclictest1377-21snmpd23:07:187
766456992417,3cyclictest0-21swapper/719:50:167
76645699240,22cyclictest0-21swapper/723:55:507
76645699240,22cyclictest0-21swapper/723:14:427
76645699240,22cyclictest0-21swapper/722:47:427
76645699240,22cyclictest0-21swapper/722:32:587
76645699240,22cyclictest0-21swapper/722:28:427
76645699240,22cyclictest0-21swapper/721:52:107
76645699240,22cyclictest0-21swapper/721:27:027
76645699240,22cyclictest0-21swapper/721:03:147
76645699240,22cyclictest0-21swapper/720:55:427
76645699240,22cyclictest0-21swapper/719:37:237
76645699240,22cyclictest0-21swapper/719:31:147
76645699240,22cyclictest0-21swapper/700:11:427
766449992422,2cyclictest0-21swapper/622:30:196
76644999240,24cyclictest0-21swapper/619:15:536
766443992422,1cyclictest1377-21snmpd23:46:255
766443992420,3cyclictest1377-21snmpd21:52:425
766443992420,3cyclictest1377-21snmpd00:31:515
766443992417,5cyclictest0-21swapper/522:28:415
76644399241,22cyclictest0-21swapper/520:27:395
76644399240,22cyclictest0-21swapper/523:00:215
766439992421,2cyclictest1377-21snmpd20:06:274
766439992421,2cyclictest1377-21snmpd20:06:274
766439992417,5cyclictest0-21swapper/421:56:254
766439992417,3cyclictest0-21swapper/423:48:314
76643999241,22cyclictest0-21swapper/422:24:224
76643999240,23cyclictest0-21swapper/420:30:584
76643999240,22cyclictest0-21swapper/422:34:114
76643999240,22cyclictest0-21swapper/422:16:404
76643999240,22cyclictest0-21swapper/420:39:284
766434992421,3cyclictest1377-21snmpd23:37:373
766434992421,3cyclictest1377-21snmpd21:24:113
766434992420,3cyclictest1377-21snmpd23:08:363
76643499240,22cyclictest0-21swapper/320:14:483
76643499240,22cyclictest0-21swapper/300:06:503
766430992423,1cyclictest0-21swapper/220:15:152
766430992422,1cyclictest1377-21snmpd22:49:592
766430992417,5cyclictest0-21swapper/222:12:222
766430992417,5cyclictest0-21swapper/220:09:212
766430992417,5cyclictest0-21swapper/220:09:202
76643099240,2cyclictest0-21swapper/222:55:502
76643099240,22cyclictest0-21swapper/222:16:462
76643099240,22cyclictest0-21swapper/219:33:202
766425992421,3cyclictest1377-21snmpd19:16:431
76642599240,2cyclictest0-21swapper/119:40:231
766421992421,3cyclictest1377-21snmpd23:33:120
766421992420,3cyclictest1377-21snmpd20:42:550
766421992417,5cyclictest0-21swapper/023:11:090
766421992417,3cyclictest0-21swapper/021:23:140
76642199240,24cyclictest0-21swapper/023:57:590
76642199240,22cyclictest0-21swapper/023:17:180
76642199240,22cyclictest0-21swapper/000:27:570
766456992321,1cyclictest1377-21snmpd22:03:437
766456992319,3cyclictest1377-21snmpd19:43:057
766456992316,5cyclictest799650-21chrt20:07:087
766456992316,5cyclictest799650-21chrt20:07:077
76645699230,2cyclictest527-21systemd-journal23:15:107
76645699230,23cyclictest0-21swapper/723:40:067
76645699230,23cyclictest0-21swapper/723:33:467
76645699230,23cyclictest0-21swapper/720:16:227
76645699230,23cyclictest0-21swapper/719:22:217
76645699230,23cyclictest0-21swapper/700:37:187
76645699230,23cyclictest0-21swapper/700:25:427
76645699230,23cyclictest0-21swapper/700:06:187
76645699230,22cyclictest0-21swapper/723:49:347
76645699230,22cyclictest0-21swapper/723:26:587
76645699230,22cyclictest0-21swapper/723:20:427
76645699230,22cyclictest0-21swapper/723:04:417
76645699230,22cyclictest0-21swapper/722:50:547
76645699230,22cyclictest0-21swapper/722:36:227
76645699230,22cyclictest0-21swapper/722:36:227
76645699230,22cyclictest0-21swapper/722:20:067
76645699230,22cyclictest0-21swapper/722:15:507
76645699230,22cyclictest0-21swapper/721:58:107
76645699230,22cyclictest0-21swapper/721:45:107
76645699230,22cyclictest0-21swapper/721:39:307
76645699230,22cyclictest0-21swapper/721:31:427
76645699230,22cyclictest0-21swapper/721:23:307
76645699230,22cyclictest0-21swapper/721:16:507
76645699230,22cyclictest0-21swapper/721:14:337
76645699230,22cyclictest0-21swapper/721:07:507
76645699230,22cyclictest0-21swapper/720:41:387
76645699230,22cyclictest0-21swapper/720:25:467
76645699230,22cyclictest0-21swapper/720:20:467
76645699230,22cyclictest0-21swapper/719:49:067
76645699230,22cyclictest0-21swapper/719:25:507
76645699230,22cyclictest0-21swapper/719:16:347
76645699230,22cyclictest0-21swapper/700:32:307
76645699230,22cyclictest0-21swapper/700:22:507
76645699230,21cyclictest0-21swapper/723:50:467
76645699230,21cyclictest0-21swapper/720:34:577
76645699230,21cyclictest0-21swapper/720:00:347
76645699230,21cyclictest0-21swapper/720:00:347
766449992319,3cyclictest1377-21snmpd19:30:116
76644999230,23cyclictest0-21swapper/623:54:086
76644999230,22cyclictest0-21swapper/623:14:306
76644999230,22cyclictest0-21swapper/621:00:196
76644999230,22cyclictest0-21swapper/620:45:196
766443992321,1cyclictest1377-21snmpd23:24:225
766443992321,1cyclictest1377-21snmpd22:49:325
766443992321,1cyclictest1377-21snmpd00:04:475
766443992320,2cyclictest1377-21snmpd23:10:475
766443992319,3cyclictest1377-21snmpd23:17:455
766443992319,3cyclictest1377-21snmpd22:07:255
766443992319,3cyclictest1377-21snmpd22:00:375
76644399230,22cyclictest0-21swapper/523:37:075
766439992321,1cyclictest1377-21snmpd23:11:094
766439992319,3cyclictest1377-21snmpd23:50:134
766439992319,3cyclictest1377-21snmpd21:17:264
766439992319,3cyclictest1377-21snmpd19:56:514
766439992317,2cyclictest0-21swapper/422:58:544
76643999230,22cyclictest0-21swapper/423:34:434
76643999230,22cyclictest0-21swapper/421:32:124
76643999230,22cyclictest0-21swapper/400:14:114
76643999230,22cyclictest0-21swapper/400:03:084
766434992322,1cyclictest0-21swapper/321:02:593
766434992319,3cyclictest1377-21snmpd23:17:333
766434992318,3cyclictest0-21swapper/300:10:163
76643499230,22cyclictest0-21swapper/323:57:563
76643499230,22cyclictest0-21swapper/321:26:143
766430992321,2cyclictest1377-21snmpd21:40:512
766430992320,2cyclictest1377-21snmpd22:30:412
766430992319,3cyclictest1377-21snmpd20:10:332
76643099231,22cyclictest0-21swapper/223:49:522
76643099230,22cyclictest0-21swapper/222:25:072
76643099230,22cyclictest0-21swapper/221:50:422
766425992321,2cyclictest1377-21snmpd22:56:171
766425992321,2cyclictest1377-21snmpd22:26:501
766425992320,3cyclictest1377-21snmpd00:15:471
766425992316,5cyclictest0-21swapper/121:55:061
76642599230,22cyclictest0-21swapper/121:52:481
76642599230,22cyclictest0-21swapper/121:09:341
76642599230,1cyclictest0-21swapper/121:25:201
76642599230,1cyclictest0-21swapper/120:20:031
766421992320,3cyclictest1377-21snmpd21:56:030
766421992320,2cyclictest1377-21snmpd23:41:460
766421992319,3cyclictest831297-21cat21:05:160
76642199230,22cyclictest0-21swapper/020:15:090
766456992216,2cyclictest0-21swapper/720:35:207
76645699220,21cyclictest0-21swapper/700:17:587
76645699220,1cyclictest0-21swapper/723:37:437
76645699220,1cyclictest0-21swapper/722:05:497
766449992220,2cyclictest0-21swapper/621:14:296
766449992220,1cyclictest1377-21snmpd23:08:296
766449992220,1cyclictest1377-21snmpd21:49:516
766449992220,1cyclictest1377-21snmpd19:41:266
766449992218,4cyclictest1377-21snmpd22:14:376
766449992217,4cyclictest1377-21snmpd00:37:036
766449992216,5cyclictest0-21swapper/620:10:066
76644999220,3cyclictest2739-21gnome-shell21:39:586
76644999220,1cyclictest0-21swapper/623:42:346
76644999220,1cyclictest0-21swapper/622:35:236
76644999220,1cyclictest0-21swapper/622:35:236
76644999220,1cyclictest0-21swapper/621:15:586
76644999220,1cyclictest0-21swapper/600:16:356
766443992220,1cyclictest1377-21snmpd23:57:355
766443992220,1cyclictest1377-21snmpd19:23:475
766443992220,1cyclictest1377-21snmpd00:35:395
766443992220,1cyclictest0-21swapper/522:38:535
766443992220,1cyclictest0-21swapper/522:38:535
766443992218,3cyclictest1377-21snmpd19:35:145
766443992216,4cyclictest0-21swapper/519:52:155
76644399220,21cyclictest0-21swapper/522:41:295
76644399220,21cyclictest0-21swapper/522:41:295
76644399220,1cyclictest0-21swapper/521:38:485
76644399220,1cyclictest0-21swapper/521:34:335
76644399220,1cyclictest0-21swapper/520:35:105
76644399220,1cyclictest0-21swapper/519:34:375
766439992217,5cyclictest0-21swapper/420:21:034
766439992215,5cyclictest0-21swapper/420:40:374
766439992215,3cyclictest0-21swapper/400:07:114
76643999220,1cyclictest59-21ksoftirqd/423:40:134
76643999220,1cyclictest0-21swapper/422:45:284
76643999220,1cyclictest0-21swapper/422:43:504
76643999220,1cyclictest0-21swapper/422:43:504
76643999220,1cyclictest0-21swapper/420:11:274
766434992221,1cyclictest1377-21snmpd23:04:573
766434992221,1cyclictest1377-21snmpd22:59:153
766434992221,1cyclictest1377-21snmpd21:46:393
766434992221,1cyclictest1377-21snmpd00:01:293
766434992219,2cyclictest1377-21snmpd19:22:503
766434992217,3cyclictest0-21swapper/300:27:333
766434992216,2cyclictest0-21swapper/321:10:173
76643499220,22cyclictest0-21swapper/322:31:103
76643499220,1cyclictest0-21swapper/319:52:033
766430992221,1cyclictest1377-21snmpd22:44:322
766430992221,1cyclictest1377-21snmpd22:44:322
766430992220,1cyclictest1377-21snmpd19:55:002
766430992218,3cyclictest1377-21snmpd23:14:242
766430992218,3cyclictest1377-21snmpd19:29:262
766430992218,3cyclictest1377-21snmpd00:17:232
766430992216,2cyclictest0-21swapper/223:35:172
76643099220,22cyclictest0-21swapper/200:29:572
76643099220,1cyclictest0-21swapper/219:48:012
76643099220,1cyclictest0-21swapper/200:39:332
766425992221,1cyclictest1377-21snmpd22:42:171
766425992221,1cyclictest1377-21snmpd22:42:171
766425992221,1cyclictest1377-21snmpd20:35:131
766425992219,3cyclictest1377-21snmpd23:44:011
766425992215,5cyclictest0-21swapper/123:27:071
766425992215,5cyclictest0-21swapper/120:56:401
766425992215,5cyclictest0-21swapper/100:02:291
76642599220,22cyclictest0-21swapper/120:09:361
76642599220,22cyclictest0-21swapper/120:09:351
76642599220,22cyclictest0-21swapper/100:32:001
76642599220,22cyclictest0-21swapper/100:05:411
76642599220,1cyclictest0-21swapper/123:19:121
76642599220,1cyclictest0-21swapper/123:12:121
76642599220,1cyclictest0-21swapper/122:50:411
76642599220,1cyclictest0-21swapper/121:45:271
76642599220,1cyclictest0-21swapper/119:25:591
766421992221,1cyclictest1377-21snmpd22:46:170
766421992221,1cyclictest1377-21snmpd00:13:050
766421992219,3cyclictest1377-21snmpd21:31:180
766421992218,3cyclictest1377-21snmpd22:35:470
766421992218,3cyclictest1377-21snmpd22:35:470
766421992218,3cyclictest1377-21snmpd22:33:080
766421992218,3cyclictest1377-21snmpd19:28:380
766421992216,5cyclictest0-21swapper/023:05:210
766421992216,4cyclictest0-21swapper/020:30:160
76642199220,22cyclictest0-21swapper/022:07:360
76642199220,22cyclictest0-21swapper/021:17:200
76642199220,22cyclictest0-21swapper/019:40:350
76642199220,1cyclictest0-21swapper/021:44:060
76642199220,1cyclictest0-21swapper/020:14:330
76642199220,1cyclictest0-21swapper/019:39:110
76642199220,1cyclictest0-21swapper/000:16:410
766449992116,4cyclictest0-21swapper/620:34:596
76644999210,20cyclictest894674-21switchtime23:00:256
766443992119,1cyclictest1377-21snmpd21:43:355
766443992119,1cyclictest1377-21snmpd19:47:115
766439992119,1cyclictest1377-21snmpd21:42:484
766439992119,1cyclictest1377-21snmpd19:38:524
766439992119,1cyclictest1377-21snmpd00:18:504
766439992117,3cyclictest1377-21snmpd22:51:504
766434992118,3cyclictest1377-21snmpd23:23:243
766434992118,3cyclictest1377-21snmpd19:11:273
766430992120,1cyclictest1377-21snmpd22:24:582
766430992117,3cyclictest1377-21snmpd23:43:432
766430992117,3cyclictest1377-21snmpd00:31:002
766430992114,5cyclictest0-21swapper/220:31:162
766425992120,1cyclictest0-21swapper/121:10:351
766425992118,3cyclictest1377-21snmpd19:34:141
766421992120,1cyclictest1377-21snmpd21:54:300
766421992117,3cyclictest1377-21snmpd00:07:320
766421992116,3cyclictest0-21swapper/022:52:440
7824782200,0sleep10-21swapper/119:35:221
766449992019,1cyclictest1377-21snmpd22:16:006
766449992019,1cyclictest1377-21snmpd21:25:476
766449992019,1cyclictest1377-21snmpd21:22:176
766449992019,1cyclictest1377-21snmpd19:13:506
766449992019,1cyclictest1377-21snmpd00:23:276
766449992019,1cyclictest0-21swapper/619:45:206
766449992016,3cyclictest1377-21snmpd21:06:026
766449992013,6cyclictest927755-21cat00:00:256
766449992013,6cyclictest775569-21grep19:24:596
76644999200,19cyclictest825670-21sh20:55:166
76644999200,16cyclictest939571-21pgrep00:25:066
766443992018,1cyclictest1377-21snmpd21:45:275
766443992018,1cyclictest1377-21snmpd00:05:445
766443992016,2cyclictest0-21swapper/522:50:155
766439992018,1cyclictest1377-21snmpd23:21:364
766439992018,1cyclictest1377-21snmpd19:40:114
766439992017,2cyclictest0-21swapper/419:15:024
766439992016,3cyclictest1377-21snmpd19:47:144
766439992014,2cyclictest0-21swapper/400:30:174
766434992019,1cyclictest1377-21snmpd21:42:093
766434992019,1cyclictest1377-21snmpd20:39:043
766430992018,2cyclictest0-21swapper/219:55:212
766430992016,3cyclictest1377-21snmpd20:45:312
766430992016,2cyclictest0-21swapper/221:15:142
766430992013,5cyclictest0-21swapper/219:13:282
766425992018,2cyclictest1377-21snmpd21:32:021
766425992017,3cyclictest1377-21snmpd23:35:191
766425992017,3cyclictest1377-21snmpd20:13:121
766425992016,3cyclictest921418-21cat23:50:161
766425992013,3cyclictest0-21swapper/121:20:171
766421992019,1cyclictest1377-21snmpd20:56:280
766421992019,1cyclictest1377-21snmpd19:53:260
766421992019,1cyclictest1377-21snmpd00:02:590
766421992018,2cyclictest1377-21snmpd20:02:540
766421992018,2cyclictest1377-21snmpd20:02:540
766449991915,2cyclictest0-21swapper/622:01:466
76644999190,19cyclictest0-21swapper/623:17:406
76644399198,10cyclictest0-21swapper/523:42:595
766443991916,2cyclictest0-21swapper/521:56:315
766443991913,4cyclictest0-21swapper/519:55:065
766439991917,1cyclictest0-21swapper/420:18:204
766439991915,3cyclictest1377-21snmpd21:45:234
766439991914,3cyclictest0-21swapper/421:07:594
766439991912,3cyclictest0-21swapper/420:51:334
766434991918,1cyclictest1377-21snmpd00:38:033
766434991918,1cyclictest0-21swapper/322:12:463
766434991912,5cyclictest0-21swapper/321:53:113
766430991918,1cyclictest1377-21snmpd20:28:142
766430991918,1cyclictest0-21swapper/221:02:582
76643099190,18cyclictest0-21swapper/222:37:592
76643099190,18cyclictest0-21swapper/222:37:582
766425991918,1cyclictest1377-21snmpd22:37:351
766425991918,1cyclictest1377-21snmpd22:37:341
766425991918,1cyclictest1377-21snmpd20:54:131
766425991915,3cyclictest1377-21snmpd23:00:061
766421991918,1cyclictest0-21swapper/023:51:040
766421991918,1cyclictest0-21swapper/020:49:550
766421991918,1cyclictest0-21swapper/019:47:560
766421991917,2cyclictest1377-21snmpd19:30:350
766421991917,1cyclictest1377-21snmpd22:02:580
766421991913,4cyclictest0-21swapper/023:23:480
76642199190,17cyclictest0-21swapper/023:00:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional