You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 18:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jun 02, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16550992721,5cyclictest23022-21apt-get09:53:420
1655099260,25cyclictest15057-21munin-run09:45:010
1655099260,1cyclictest0-21swapper/012:29:380
1656599250,1cyclictest15143-21ssh11:47:233
16560992521,1cyclictest0-21swapper/211:55:422
1656099250,24cyclictest10237-21diskmemload10:58:472
1656099250,24cyclictest10237-21diskmemload09:23:522
1656099250,24cyclictest0-21swapper/211:20:282
1655599250,24cyclictest19829-21ssh09:49:211
1655599250,24cyclictest10237-21diskmemload11:49:271
1655599250,24cyclictest10237-21diskmemload09:36:321
1655599250,24cyclictest10237-21diskmemload09:36:321
16550992522,3cyclictest0-21swapper/009:29:160
16550992521,3cyclictest0-21swapper/011:26:440
16550992520,4cyclictest1831-21ssh12:34:460
1655099250,24cyclictest10237-21diskmemload12:35:380
1655099250,24cyclictest0-21swapper/012:09:190
1655099250,24cyclictest0-21swapper/009:10:120
1655099250,22cyclictest62550irq/127-eno110:36:370
1655099250,0cyclictest0-21swapper/010:13:430
16565992422,1cyclictest7849-21cron09:05:013
1656599242,1cyclictest3288-21kworker/u8:112:07:203
1656599240,24cyclictest0-21swapper/311:56:213
1656599240,23cyclictest0-21swapper/310:26:203
1656599240,1cyclictest31555-21rm11:32:213
1656599240,1cyclictest0-21swapper/311:25:293
16560992423,1cyclictest24939-21ssh12:26:262
1656099240,24cyclictest0-21swapper/210:32:092
1656099240,23cyclictest10237-21diskmemload09:38:282
1656099240,23cyclictest10237-21diskmemload09:38:282
1656099240,23cyclictest0-21swapper/209:42:502
1656099240,0cyclictest0-21swapper/210:26:082
1656099240,0cyclictest0-21swapper/209:55:042
16555992424,0cyclictest0-21swapper/109:11:481
1655599240,24cyclictest0-21swapper/110:49:311
1655599240,1cyclictest0-21swapper/111:01:411
16550992421,3cyclictest0-21swapper/011:21:280
16550992421,2cyclictest16614-21grep11:48:440
16550992420,3cyclictest10237-21diskmemload12:21:490
16550992420,3cyclictest0-21swapper/009:35:590
16550992420,3cyclictest0-21swapper/009:35:590
1655099240,24cyclictest0-21swapper/011:08:070
1655099240,23cyclictest18448-21ssh12:19:590
1655099240,23cyclictest12994-21sh11:14:170
1655099240,23cyclictest12994-21sh11:14:170
1655099240,23cyclictest0-21swapper/011:17:380
1655099240,21cyclictest0-21swapper/011:37:290
1655099240,0cyclictest0-21swapper/010:58:450
1655099240,0cyclictest0-21swapper/010:29:230
16565992323,0cyclictest0-21swapper/312:17:393
16565992323,0cyclictest0-21swapper/311:41:203
16565992323,0cyclictest0-21swapper/311:20:563
16565992323,0cyclictest0-21swapper/311:13:063
16565992323,0cyclictest0-21swapper/311:13:063
16565992323,0cyclictest0-21swapper/310:11:013
16565992322,1cyclictest10237-21diskmemload12:03:443
16565992322,1cyclictest0-21swapper/312:10:373
1656599230,23cyclictest10237-21diskmemload12:24:113
1656599230,23cyclictest10237-21diskmemload09:50:233
1656599230,23cyclictest10237-21diskmemload09:31:323
1656599230,23cyclictest0-21swapper/312:36:383
1656599230,23cyclictest0-21swapper/311:39:053
1656599230,23cyclictest0-21swapper/310:33:113
1656599230,23cyclictest0-21swapper/310:07:003
1656599230,22cyclictest0-21swapper/310:01:183
1656599230,1cyclictest18817-21perf11:20:013
1656599230,0cyclictest0-21swapper/309:08:423
16560992323,0cyclictest0-21swapper/212:03:272
16560992323,0cyclictest0-21swapper/211:07:372
16560992323,0cyclictest0-21swapper/209:50:352
16560992321,1cyclictest23351-21ssh12:24:292
16560992321,1cyclictest0-21swapper/212:16:492
16560992321,1cyclictest0-21swapper/211:39:592
16560992321,1cyclictest0-21swapper/211:31:412
16560992320,2cyclictest20683-21idleruntime-cro08:25:002
16560992318,4cyclictest0-21swapper/210:10:142
16560992318,4cyclictest0-21swapper/209:15:492
1656099230,2cyclictest18282-21sendmail-msp08:20:012
1656099230,22cyclictest4513-21ssh12:07:462
1656099230,22cyclictest20422-21ssh11:52:262
1656099230,22cyclictest15504-21rm11:17:462
1656099230,22cyclictest10237-21diskmemload10:05:272
1656099230,22cyclictest10237-21diskmemload09:10:342
1656099230,1cyclictest15020-21ssh11:47:132
1656099230,0cyclictest0-21swapper/212:12:472
1656099230,0cyclictest0-21swapper/210:40:502
1656099230,0cyclictest0-21swapper/209:47:512
16555992323,0cyclictest10237-21diskmemload11:39:231
16555992323,0cyclictest0-21swapper/111:55:081
16555992323,0cyclictest0-21swapper/111:11:061
16555992323,0cyclictest0-21swapper/111:11:061
16555992323,0cyclictest0-21swapper/111:06:271
16555992321,1cyclictest783-21snmpd12:01:381
16555992321,1cyclictest13099-21latency_hist12:15:011
16555992320,2cyclictest25050-21sh12:26:361
1655599230,2cyclictest19972-21ssh11:21:171
1655599230,23cyclictest0-21swapper/109:57:261
1655599230,22cyclictest31865-21sh09:30:111
1655599230,22cyclictest2941-21ssh10:34:441
1655599230,22cyclictest20389-21sh11:52:231
1655599230,22cyclictest18796-21dump-pmu-power11:20:011
1655599230,22cyclictest10237-21diskmemload10:22:071
1655599230,22cyclictest10237-21diskmemload10:15:221
1655599230,22cyclictest10237-21diskmemload09:22:371
1655599230,22cyclictest0-21swapper/110:55:241
1655599230,22cyclictest0-21swapper/110:52:251
1655599230,1cyclictest0-21swapper/111:42:341
1655599230,0cyclictest0-21swapper/110:06:001
16550992323,0cyclictest0-21swapper/010:30:110
16550992323,0cyclictest0-21swapper/010:20:490
1655099230,2cyclictest8282-21sh12:10:230
1655099230,2cyclictest26601-21sh11:58:420
1655099230,23cyclictest0-21swapper/009:15:200
1655099230,22cyclictest5436-21sh09:00:010
1655099230,22cyclictest31320-21rm11:32:010
1655099230,22cyclictest15633-21ssh10:16:430
1655099230,22cyclictest12791-21open_inodes10:43:490
1655099230,22cyclictest10614-21apt-get08:05:020
1655099230,22cyclictest10237-21diskmemload09:55:100
1655099230,21cyclictest10237-21diskmemload11:40:210
1655099230,1cyclictest0-21swapper/010:47:330
1655099230,1cyclictest0-21swapper/010:07:320
1655099230,0cyclictest10237-21diskmemload10:50:040
1655099230,0cyclictest0-21swapper/012:00:340
16565992222,0cyclictest10237-21diskmemload10:51:503
16565992222,0cyclictest10237-21diskmemload10:20:323
16565992222,0cyclictest10237-21diskmemload09:49:573
16565992222,0cyclictest0-21swapper/310:41:393
16565992221,1cyclictest25878-21idleruntime-cro08:35:003
1656599220,22cyclictest10237-21diskmemload09:14:043
1656599220,22cyclictest0-21swapper/312:26:403
1656599220,22cyclictest0-21swapper/311:00:133
1656599220,22cyclictest0-21swapper/310:45:153
1656599220,21cyclictest13939-21apt-get08:13:413
1656599220,21cyclictest0-21swapper/309:16:163
1656599220,1cyclictest4700-21ssh09:35:113
1656599220,1cyclictest4700-21ssh09:35:113
1656599220,1cyclictest30667-21ssh12:32:043
1656599220,1cyclictest10903-21ssh09:41:333
1656599220,0cyclictest10237-21diskmemload10:55:253
1656599220,0cyclictest10237-21diskmemload09:28:433
1656599220,0cyclictest0-21swapper/311:53:483
1656599220,0cyclictest0-21swapper/310:36:343
1656599220,0cyclictest0-21swapper/309:59:513
1656599220,0cyclictest0-21swapper/309:20:133
16560992222,0cyclictest10237-21diskmemload09:30:192
16560992222,0cyclictest0-21swapper/211:14:122
16560992222,0cyclictest0-21swapper/211:14:122
16560992222,0cyclictest0-21swapper/210:37:492
16560992221,1cyclictest10237-21diskmemload11:27:242
16560992221,0cyclictest0-21swapper/210:15:242
16560992219,3cyclictest0-21swapper/210:51:032
1656099221,1cyclictest24399-21kworker/u8:012:31:042
1656099220,21cyclictest7842-21idleruntime-cro09:05:012
1656099220,21cyclictest2707-21ssh12:35:482
1656099220,21cyclictest19999-21ssh10:20:092
1656099220,21cyclictest10237-21diskmemload09:25:562
1656099220,21cyclictest0-21swapper/211:40:042
1656099220,21cyclictest0-21swapper/211:00:512
1656099220,1cyclictest1-21systemd07:15:012
1656099220,1cyclictest10237-21diskmemload10:03:522
16555992222,0cyclictest0-21swapper/112:16:401
16555992222,0cyclictest0-21swapper/109:40:131
1655599220,22cyclictest0-21swapper/112:38:011
1655599220,22cyclictest0-21swapper/112:31:511
1655599220,21cyclictest6340-21apt-get07:58:411
1655599220,21cyclictest26881-21ssh09:25:451
1655599220,21cyclictest10237-21diskmemload11:25:151
1655599220,21cyclictest0-21swapper/110:41:291
1655599220,21cyclictest0-21swapper/110:35:431
1655599220,21cyclictest0-21swapper/109:50:351
1655599220,1cyclictest5193-21apt-get12:08:421
1655599220,1cyclictest10614-21apt-get08:05:011
1655599220,1cyclictest0-21swapper/112:20:011
1655599220,0cyclictest0-21swapper/109:03:401
16550992221,1cyclictest21182-21ssh09:20:090
16550992221,1cyclictest10237-21diskmemload10:00:260
16550992220,2cyclictest0-21swapper/009:45:100
16550992220,1cyclictest31564-21apt-get07:43:400
16550992220,1cyclictest23037-21cron08:30:000
16550992215,6cyclictest10622-21munin-run08:05:010
1655099220,22cyclictest10237-21diskmemload09:32:200
1655099220,1cyclictest18818-21apt-get08:23:420
1655099220,1cyclictest0-21swapper/011:52:230
16565992121,0cyclictest0-21swapper/310:15:323
1656599210,20cyclictest1186-21apt-get08:53:423
1656599210,20cyclictest0-21swapper/308:18:393
1656599210,0cyclictest26403-21apt-get08:38:413
1656599210,0cyclictest0-21swapper/308:43:423
16560992120,1cyclictest31564-21apt-get07:43:402
16560992119,1cyclictest0-21swapper/207:48:422
1656099210,20cyclictest0-21swapper/208:30:022
1656099210,1cyclictest8756-21apt-get08:03:412
1656099210,0cyclictest0-21swapper/210:45:212
16555992121,0cyclictest10237-21diskmemload10:10:341
1655599210,20cyclictest0-21swapper/108:33:421
1655599210,0cyclictest0-21swapper/108:58:421
1655599210,0cyclictest0-21swapper/107:18:411
16550992120,1cyclictest18251-21perf08:20:010
1655099210,1cyclictest3416-21idleruntime-cro07:50:000
1655099210,0cyclictest0-21swapper/011:00:180
16565992020,0cyclictest3593-21apt-get08:58:423
16565992020,0cyclictest21208-21apt-get08:28:413
16565992020,0cyclictest0-21swapper/308:23:423
16565992020,0cyclictest0-21swapper/307:13:403
16565992019,0cyclictest0-21swapper/307:28:413
1656599200,20cyclictest0-21swapper/307:53:423
1656599200,0cyclictest26653-21apt-get07:33:423
16560992020,0cyclictest0-21swapper/208:53:422
16560992020,0cyclictest0-21swapper/207:38:422
16560992020,0cyclictest0-21swapper/207:33:422
16560992018,1cyclictest21858-21apt-get07:23:412
16560992017,2cyclictest8203-21latency_hist08:00:002
1656099200,20cyclictest0-21swapper/208:38:422
1656099200,20cyclictest0-21swapper/207:53:422
1656099200,0cyclictest28819-21apt-get08:43:422
16555992020,0cyclictest8402-21apt-get09:08:421
16555992020,0cyclictest0-21swapper/108:28:411
16555992020,0cyclictest0-21swapper/107:28:411
16555992019,0cyclictest0-21swapper/108:38:421
16550992018,1cyclictest24003-21apt-get08:33:410
1655099200,20cyclictest0-21swapper/008:13:410
16565991919,0cyclictest0-21swapper/308:05:023
16565991919,0cyclictest0-21swapper/307:48:423
16565991918,1cyclictest8226-21awk08:00:013
16560991919,0cyclictest0-21swapper/208:08:392
16555991919,0cyclictest0-21swapper/107:23:411
16550991919,0cyclictest3947-21apt-get07:53:420
16550991919,0cyclictest0-21swapper/009:08:420
16550991919,0cyclictest0-21swapper/007:58:410
16550991919,0cyclictest0-21swapper/007:28:410
16550991919,0cyclictest0-21swapper/007:23:410
16550991919,0cyclictest0-21swapper/007:13:400
1655099190,6cyclictest0-21swapper/007:33:410
16565991816,1cyclictest994-21turbostat07:45:003
16560991815,2cyclictest15786-21cron08:15:012
16555991818,0cyclictest0-21swapper/108:53:431
16555991818,0cyclictest0-21swapper/108:18:391
16565991717,0cyclictest0-21swapper/308:03:403
16565991717,0cyclictest0-21swapper/307:18:413
16560991716,0cyclictest24251-21apt-get07:28:412
1655599171,15cyclictest3325-21munin-node08:20:581
16555991616,0cyclictest11541-21apt-get08:08:391
16555991614,1cyclictest0-21swapper/107:38:431
1655599160,15cyclictest17843-21irqstats07:13:441
1655099162,13cyclictest0-21swapper/007:18:410
16550991616,0cyclictest0-21swapper/007:38:430
1655099160,16cyclictest0-21swapper/008:42:330
1656099150,1cyclictest0-21swapper/207:18:452
16555991515,0cyclictest0-21swapper/107:53:421
1655599150,1cyclictest0-21swapper/108:50:011
16550991515,0cyclictest62550irq/127-eno108:51:290
16550991514,1cyclictest0-21swapper/009:02:590
1656599140,14cyclictest0-21swapper/308:45:423
1656599140,14cyclictest0-21swapper/307:21:083
1656599140,0cyclictest0-21swapper/307:37:223
1656099140,14cyclictest0-21swapper/208:46:132
1656099140,14cyclictest0-21swapper/208:26:302
1656099140,0cyclictest0-21swapper/209:05:512
16555991414,0cyclictest0-21swapper/107:46:021
16555991414,0cyclictest0-21swapper/107:40:091
1655599140,0cyclictest0-21swapper/108:40:091
16550991414,0cyclictest0-21swapper/008:36:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional