You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 17:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jun 23, 2026 12:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15194992524,1cyclictest3861-21ssh10:02:373
15190992523,1cyclictest0-21swapper/211:30:092
15186992523,1cyclictest0-21swapper/110:14:471
1518699250,1cyclictest0-21swapper/111:42:561
1518699250,0cyclictest5173-21ssh11:59:221
1518299252,22cyclictest62550irq/127-eno109:15:380
1518299252,22cyclictest27812-21sh11:50:270
1518299252,1cyclictest62550irq/127-eno110:47:260
1518299250,1cyclictest62550irq/127-eno111:30:090
15194992422,1cyclictest313-21systemd-journal10:30:013
1519499240,24cyclictest8938-21diskmemload11:17:443
1519499240,23cyclictest26450-21ssh12:17:443
1519499240,0cyclictest0-21swapper/310:54:553
1519099241,22cyclictest18473-21sh10:15:332
1519099240,24cyclictest0-21swapper/212:23:522
1519099240,24cyclictest0-21swapper/211:44:072
1519099240,23cyclictest8938-21diskmemload12:36:452
1519099240,23cyclictest8938-21diskmemload11:01:152
1519099240,0cyclictest8938-21diskmemload10:59:222
15186992422,1cyclictest0-21swapper/112:02:581
1518699240,24cyclictest8938-21diskmemload11:50:301
1518699240,24cyclictest0-21swapper/112:23:521
1518699240,24cyclictest0-21swapper/109:18:001
1518699240,23cyclictest5975-21ssh11:01:121
1518699240,23cyclictest0-21swapper/111:35:421
1518699240,23cyclictest0-21swapper/110:45:291
1518699240,23cyclictest0-21swapper/110:25:501
1518699240,23cyclictest0-21swapper/110:02:401
1518699240,22cyclictest313-21systemd-journal09:15:001
15182992423,1cyclictest0-21swapper/010:00:580
15182992421,1cyclictest8938-21diskmemload10:57:330
1518299240,23cyclictest9050-21diskmemload10:35:420
1518299240,23cyclictest0-21swapper/009:47:030
15194992323,0cyclictest8938-21diskmemload11:31:383
15194992323,0cyclictest0-21swapper/311:06:583
15194992323,0cyclictest0-21swapper/311:01:273
15194992323,0cyclictest0-21swapper/310:20:053
15194992322,1cyclictest8938-21diskmemload10:06:563
15194992321,1cyclictest0-21swapper/309:45:383
1519499231,1cyclictest9303-21ssh09:10:093
1519499230,2cyclictest20190-21ssh11:43:423
1519499230,2cyclictest12548-21ssh09:40:523
1519499230,23cyclictest8938-21diskmemload12:28:243
1519499230,23cyclictest8938-21diskmemload09:35:163
1519499230,23cyclictest0-21swapper/310:56:293
1519499230,22cyclictest4085-21sh10:31:283
1519499230,22cyclictest27087-21ssh09:55:053
1519499230,22cyclictest23830-21ssh11:45:533
1519499230,22cyclictest20244-21ssh12:11:523
1519499230,22cyclictest16951-21ssh09:15:513
1519499230,22cyclictest13051-21rm11:36:303
1519499230,22cyclictest11820-21ssh12:05:263
1519499230,1cyclictest8607-21switchtime10:35:353
1519499230,0cyclictest8938-21diskmemload09:50:253
1519499230,0cyclictest0-21swapper/310:40:353
1519499230,0cyclictest0-21swapper/310:13:143
15190992323,0cyclictest8938-21diskmemload11:23:452
15190992323,0cyclictest0-21swapper/212:27:242
15190992322,1cyclictest8938-21diskmemload10:30:062
15190992322,1cyclictest18572-21ssh09:18:052
15190992321,2cyclictest17028-21ssh11:10:542
15190992320,2cyclictest18531-21ssh09:46:272
1519099230,2cyclictest24326-21ssh11:18:222
1519099230,23cyclictest8938-21diskmemload12:32:102
1519099230,23cyclictest8938-21diskmemload10:51:072
1519099230,23cyclictest0-21swapper/209:40:202
1519099230,23cyclictest0-21swapper/209:33:002
1519099230,22cyclictest8938-21diskmemload10:05:192
1519099230,22cyclictest8938-21diskmemload09:52:042
1519099230,22cyclictest7338-21ssh09:36:182
1519099230,22cyclictest26145-21ssh11:49:062
1519099230,22cyclictest14397-21cron09:15:002
1519099230,22cyclictest14078-21cron08:15:012
1519099230,22cyclictest11717-21ssh11:06:202
1519099230,22cyclictest0-21swapper/211:35:422
1519099230,0cyclictest8938-21diskmemload10:42:052
1519099230,0cyclictest8938-21diskmemload09:55:532
1519099230,0cyclictest0-21swapper/212:13:172
1519099230,0cyclictest0-21swapper/212:05:582
15186992323,0cyclictest0-21swapper/111:06:271
1518699232,21cyclictest8917-21kworker/1:211:20:411
15186992322,0cyclictest0-21swapper/108:25:261
15186992321,1cyclictest1557-21sh12:25:021
15186992321,1cyclictest0-21swapper/112:36:101
15186992321,1cyclictest0-21swapper/110:41:211
15186992321,1cyclictest0-21swapper/109:28:241
15186992321,1cyclictest0-21swapper/109:21:181
15186992320,2cyclictest4487-21sh07:55:001
1518699230,2cyclictest18078-21grep12:10:291
1518699230,23cyclictest0-21swapper/109:33:001
1518699230,22cyclictest8938-21diskmemload09:45:251
1518699230,22cyclictest606-21dbus-daemon08:15:011
1518699230,22cyclictest26442-21ssh10:50:581
1518699230,22cyclictest17833-21apt-get07:15:251
1518699230,22cyclictest0-21swapper/112:30:491
1518699230,22cyclictest0-21swapper/111:15:341
1518699230,22cyclictest0-21swapper/110:55:561
1518699230,22cyclictest0-21swapper/110:19:281
1518699230,0cyclictest9968-21ssh11:35:001
1518699230,0cyclictest0-21swapper/112:05:581
1518699230,0cyclictest0-21swapper/109:55:531
15182992323,0cyclictest0-21swapper/011:57:370
15182992323,0cyclictest0-21swapper/011:10:540
15182992323,0cyclictest0-21swapper/011:04:270
15182992323,0cyclictest0-21swapper/010:20:530
15182992323,0cyclictest0-21swapper/009:30:380
1518299232,21cyclictest62550irq/127-eno110:41:370
15182992321,1cyclictest8938-21diskmemload11:20:500
15182992321,1cyclictest6192-21apt-get12:00:260
15182992321,1cyclictest0-21swapper/012:15:510
15182992321,1cyclictest0-21swapper/010:15:310
15182992321,1cyclictest0-21swapper/010:05:340
15182992321,1cyclictest0-21swapper/009:25:560
15182992321,1cyclictest0-21swapper/009:00:240
1518299231,1cyclictest13-21ksoftirqd/011:15:340
1518299230,2cyclictest1745-21latency_hist08:55:010
1518299230,23cyclictest0-21swapper/011:29:190
1518299230,23cyclictest0-21swapper/009:50:250
1518299230,22cyclictest9596-21diskmemload09:39:300
1518299230,22cyclictest8938-21diskmemload11:49:530
1518299230,22cyclictest8938-21diskmemload10:11:430
1518299230,22cyclictest8255-21if_enp1s012:30:300
1518299230,22cyclictest0-21swapper/009:55:440
1518299230,22cyclictest0-21swapper/009:41:110
1518299230,0cyclictest0-21swapper/011:06:270
1518299230,0cyclictest0-21swapper/010:30:060
15194992222,0cyclictest0-21swapper/310:16:153
15194992222,0cyclictest0-21swapper/309:31:013
15194992222,0cyclictest0-21swapper/309:25:013
15194992220,1cyclictest6568-21grep09:05:013
1519499220,21cyclictest8938-21diskmemload12:20:083
1519499220,21cyclictest7528-21ssh12:30:263
1519499220,21cyclictest606-21dbus-daemon08:35:003
1519499220,21cyclictest28615-21ssh11:21:303
1519499220,21cyclictest0-21swapper/311:25:323
1519499220,21cyclictest0-21swapper/309:22:263
1519499220,1cyclictest18073-21ssh10:45:053
1519499220,1cyclictest0-21swapper/311:10:263
1519499220,0cyclictest8938-21diskmemload12:03:403
1519499220,0cyclictest8938-21diskmemload11:56:013
15190992222,0cyclictest0-21swapper/212:02:582
15190992222,0cyclictest0-21swapper/210:36:512
15190992222,0cyclictest0-21swapper/208:25:262
15190992221,1cyclictest8938-21diskmemload11:25:322
15190992220,1cyclictest14525-21ssh10:11:512
1519099220,22cyclictest0-21swapper/209:27:022
1519099220,21cyclictest25093-21diskmemload10:21:042
1519099220,21cyclictest23378-21ssh09:21:592
1519099220,1cyclictest30721-21ssh10:26:222
1519099220,1cyclictest25152-21ssh12:15:582
1519099220,1cyclictest0-21swapper/211:50:272
1519099220,1cyclictest0-21swapper/210:00:282
1519099220,0cyclictest8938-21diskmemload11:56:342
1519099220,0cyclictest8938-21diskmemload10:45:292
15186992222,0cyclictest0-21swapper/111:48:211
15186992222,0cyclictest0-21swapper/110:36:511
15186992222,0cyclictest0-21swapper/110:30:061
15186992222,0cyclictest0-21swapper/109:50:251
15186992221,1cyclictest0-21swapper/110:20:021
15186992221,0cyclictest24400-21apt-get08:35:271
15186992220,1cyclictest16509-21cron08:20:011
15186992220,1cyclictest0-21swapper/112:15:381
1518699220,21cyclictest0-21swapper/111:25:301
1518699220,21cyclictest0-21swapper/109:40:251
1518699220,21cyclictest0-21swapper/107:10:251
1518699220,1cyclictest4724-21ssh09:35:041
1518699220,1cyclictest32115-21turbostat07:45:001
1518699220,1cyclictest1742-21cron08:55:011
1518699220,1cyclictest0-21swapper/111:10:341
1518699220,0cyclictest0-21swapper/110:05:191
1518699220,0cyclictest0-21swapper/108:20:251
1518299222,20cyclictest25740-21kworker/0:007:30:260
15182992221,1cyclictest1897-21apt-get12:25:260
15182992220,1cyclictest22377-21munin-run07:25:010
15182992220,1cyclictest0-21swapper/010:26:200
1518299220,22cyclictest0-21swapper/011:41:460
1518299220,21cyclictest62550irq/127-eno112:10:210
1518299220,21cyclictest4498-21ls07:55:000
1518299220,21cyclictest24760-21perf07:30:010
1518299220,21cyclictest15062-21sh12:35:410
1518299220,21cyclictest0-21swapper/010:50:020
1518299220,21cyclictest0-21swapper/009:20:030
1518299220,1cyclictest12577-21sh11:35:460
1518299220,0cyclictest0-21swapper/008:25:260
15194992120,1cyclictest7206-21apt-get08:00:253
15194992120,1cyclictest28923-21apt-get08:45:023
15194992119,1cyclictest26539-21idleruntime-cro08:40:003
15194992119,1cyclictest19606-21apt-get08:25:273
1519499210,21cyclictest0-21swapper/307:45:263
1519499210,0cyclictest8938-21diskmemload12:35:283
15190992120,1cyclictest606-21dbus-daemon07:55:002
15190992119,1cyclictest16488-21sendmail08:20:012
1519099210,21cyclictest0-21swapper/207:15:252
1519099210,20cyclictest0-21swapper/207:25:252
1519099210,1cyclictest1744-21dump-pmu-power08:55:012
15186992119,1cyclictest22002-21apt-get08:30:241
15182992120,0cyclictest0-21swapper/008:40:250
15182992118,3cyclictest0-21swapper/008:05:260
1518299210,20cyclictest0-21swapper/012:05:280
1518299210,0cyclictest0-21swapper/008:35:270
15194992020,0cyclictest0-21swapper/307:15:253
1519499200,0cyclictest0-21swapper/308:20:253
1519499200,0cyclictest0-21swapper/308:10:263
15190992020,0cyclictest6848-21apt-get09:05:242
15190992020,0cyclictest0-21swapper/208:35:272
15190992019,0cyclictest4452-21apt-get09:00:242
15190992018,1cyclictest26813-21apt-get08:40:252
15190992018,1cyclictest0-21swapper/207:25:012
1519099200,0cyclictest29606-21apt-get08:45:272
1519099200,0cyclictest27439-21apt-get07:35:252
1519099200,0cyclictest0-21swapper/208:20:252
15186992019,0cyclictest32409-21apt-get07:45:261
1518699200,20cyclictest0-21swapper/107:25:251
15182992018,1cyclictest14390-21apt-get08:15:260
1518299200,20cyclictest0-21swapper/007:45:260
1518299200,0cyclictest0-21swapper/008:30:240
15194991919,0cyclictest0-21swapper/308:50:253
15194991918,1cyclictest22375-21perf07:25:013
15194991917,1cyclictest29658-21perf07:40:003
15190991919,0cyclictest0-21swapper/208:30:242
15190991919,0cyclictest0-21swapper/207:45:262
15190991918,1cyclictest25042-21apt-get07:30:262
15186991919,0cyclictest0-21swapper/109:00:241
15186991919,0cyclictest0-21swapper/107:35:251
15186991919,0cyclictest0-21swapper/107:20:241
1518699190,19cyclictest0-21swapper/108:45:361
15182991919,0cyclictest0-21swapper/007:40:250
15182991919,0cyclictest0-21swapper/007:10:250
15182991918,1cyclictest0-21swapper/007:15:250
15182991918,0cyclictest0-21swapper/009:05:230
15194991818,0cyclictest0-21swapper/309:05:243
15186991818,0cyclictest0-21swapper/108:40:251
15186991818,0cyclictest0-21swapper/107:30:261
1518299180,1cyclictest0-21swapper/008:45:520
15194991716,1cyclictest22647-21apt-get07:25:253
15194991716,1cyclictest15178-21cyclictest07:40:253
15190991717,0cyclictest15438-21apt-get07:10:252
15186991716,0cyclictest9601-21apt-get08:05:261
15182991717,0cyclictest2036-21apt-get08:55:240
15182991716,1cyclictest16813-21apt-get08:20:250
15182991714,2cyclictest0-21swapper/008:10:270
15194991614,1cyclictest4110-21users07:50:353
15190991616,0cyclictest0-21swapper/208:05:262
15190991616,0cyclictest0-21swapper/207:40:252
15186991615,1cyclictest0-21swapper/108:55:241
1519499150,1cyclictest26761-21users07:30:353
1519499150,1cyclictest16295-21latency07:10:293
1519499150,14cyclictest0-21swapper/308:40:323
1519099150,14cyclictest0-21swapper/208:00:252
1518699150,14cyclictest0-21swapper/108:00:251
15194991414,0cyclictest0-21swapper/308:55:353
1519499140,14cyclictest0-21swapper/308:06:553
1519499140,0cyclictest0-21swapper/308:17:473
1519499140,0cyclictest0-21swapper/307:56:533
15190991414,0cyclictest0-21swapper/208:55:242
15182991414,0cyclictest0-21swapper/007:36:140
1518299140,14cyclictest0-21swapper/008:01:520
15190991313,0cyclictest0-21swapper/207:55:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional