You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 13:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 09, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3963992725,1cyclictest0-21swapper/121:40:401
3961992625,1cyclictest62550irq/127-eno122:12:360
3971992523,1cyclictest0-21swapper/321:40:403
397199250,2cyclictest31254-21diskmemload22:15:053
397199250,24cyclictest30809-21rm00:01:223
3961992525,0cyclictest0-21swapper/000:00:100
396199250,24cyclictest0-21swapper/022:41:580
396199250,24cyclictest0-21swapper/021:53:070
3971992423,1cyclictest23624-21ssh22:29:183
3971992423,1cyclictest15914-21sh21:53:063
397199240,1cyclictest11550-21ssh21:49:473
3966992423,1cyclictest31254-21diskmemload23:09:172
3966992423,1cyclictest15931-21ssh21:53:072
3966992422,2cyclictest30584-21gltestperf22:06:442
396699240,24cyclictest31254-21diskmemload22:45:582
396699240,24cyclictest0-21swapper/200:36:212
396699240,23cyclictest31254-21diskmemload21:15:382
396699240,23cyclictest0-21swapper/223:15:132
396699240,1cyclictest4380-21ssh21:43:162
396699240,0cyclictest19132-21ssh22:52:372
3963992423,1cyclictest0-21swapper/122:55:541
3963992423,1cyclictest0-21swapper/121:55:571
3963992421,2cyclictest16658-21latency_hist23:20:001
396399240,2cyclictest15398-21/usr/sbin/munin00:16:491
396399240,24cyclictest0-21swapper/122:06:231
396399240,24cyclictest0-21swapper/100:08:201
396399240,1cyclictest0-21swapper/122:18:091
396399240,1cyclictest0-21swapper/100:01:221
396399240,0cyclictest0-21swapper/122:29:181
396399240,0cyclictest0-21swapper/121:37:331
3961992422,2cyclictest30494-21diskmemload00:29:310
3961992422,1cyclictest0-21swapper/021:47:450
396199241,23cyclictest62550irq/127-eno123:44:150
396199240,24cyclictest31254-21diskmemload21:14:090
396199240,23cyclictest6263-21diskmemload23:39:010
396199240,23cyclictest30583-21tail22:06:440
396199240,23cyclictest0-21swapper/023:56:230
396199240,23cyclictest0-21swapper/000:34:230
396199240,22cyclictest0-21swapper/022:16:050
396199240,1cyclictest31254-21diskmemload21:59:430
396199240,1cyclictest16574-21ssh21:26:000
396199240,1cyclictest0-21swapper/023:15:130
3971992323,0cyclictest31254-21diskmemload22:06:233
3971992323,0cyclictest31254-21diskmemload21:59:253
3971992323,0cyclictest0-21swapper/322:00:313
3971992323,0cyclictest0-21swapper/321:37:333
3971992323,0cyclictest0-21swapper/300:37:043
3971992323,0cyclictest0-21swapper/300:08:203
397199230,2cyclictest6510-21sh23:11:163
397199230,2cyclictest25312-21ssh22:31:173
397199230,2cyclictest25096-21sh22:58:023
397199230,2cyclictest16431-21ssh23:19:383
397199230,23cyclictest31254-21diskmemload23:53:123
397199230,23cyclictest0-21swapper/323:31:413
397199230,23cyclictest0-21swapper/322:20:483
397199230,22cyclictest7992-21ssh23:40:533
397199230,22cyclictest24495-21ssh23:55:323
397199230,22cyclictest23418-21ssh00:22:373
397199230,22cyclictest22823-21ssh23:25:253
397199230,22cyclictest13823-21ssh21:22:263
397199230,0cyclictest31254-21diskmemload23:37:563
3966992323,0cyclictest0-21swapper/222:40:222
3966992323,0cyclictest0-21swapper/221:14:092
3966992323,0cyclictest0-21swapper/200:00:102
3966992322,1cyclictest31254-21diskmemload22:59:572
3966992322,1cyclictest31254-21diskmemload21:22:422
3966992321,1cyclictest0-21swapper/223:56:232
3966992320,2cyclictest17585-21ssh22:23:482
396699231,21cyclictest30319-21ssh22:35:332
396699230,23cyclictest0-21swapper/223:04:412
396699230,23cyclictest0-21swapper/221:32:022
396699230,23cyclictest0-21swapper/221:26:002
396699230,22cyclictest9795-21sh21:47:252
396699230,22cyclictest26218-21ssh00:26:062
396699230,22cyclictest21908-21rm23:52:172
396699230,22cyclictest13588-21ssh23:45:502
396699230,22cyclictest12537-21sh23:44:452
396699230,0cyclictest31254-21diskmemload22:31:592
396699230,0cyclictest0-21swapper/223:14:302
396699230,0cyclictest0-21swapper/222:12:022
3963992323,0cyclictest0-21swapper/123:05:541
3963992323,0cyclictest0-21swapper/122:48:571
3963992323,0cyclictest0-21swapper/121:31:181
3963992323,0cyclictest0-21swapper/100:12:051
3963992322,1cyclictest0-21swapper/121:21:421
3963992321,1cyclictest0-21swapper/123:21:091
3963992321,1cyclictest0-21swapper/122:52:271
396399230,2cyclictest16451-21basename19:35:001
396399230,23cyclictest31254-21diskmemload23:00:551
396399230,22cyclictest27170-21rm22:03:161
396399230,22cyclictest18554-21perf22:25:001
396399230,22cyclictest16920-21sh23:48:081
396399230,22cyclictest0-21swapper/123:41:461
396399230,0cyclictest31254-21diskmemload23:35:481
396399230,0cyclictest0-21swapper/123:28:471
396399230,0cyclictest0-21swapper/121:11:371
396399230,0cyclictest0-21swapper/100:37:041
396399230,0cyclictest0-21swapper/100:33:211
3961992322,1cyclictest0-21swapper/023:11:410
3961992321,1cyclictest0-21swapper/022:58:430
3961992321,1cyclictest0-21swapper/021:20:230
3961992320,3cyclictest0-21swapper/021:18:170
396199230,2cyclictest19789-21diskmemload22:53:340
396199230,23cyclictest31254-21diskmemload23:05:400
396199230,23cyclictest0-21swapper/023:25:080
396199230,23cyclictest0-21swapper/023:03:510
396199230,23cyclictest0-21swapper/022:00:210
396199230,23cyclictest0-21swapper/021:41:310
396199230,22cyclictest31254-21diskmemload00:06:130
396199230,22cyclictest30240-21diskmemload22:35:280
396199230,22cyclictest28885-21perf21:05:000
396199230,22cyclictest0-21swapper/022:49:390
396199230,22cyclictest0-21swapper/021:30:510
396199230,22cyclictest0-21swapper/000:35:420
396199230,1cyclictest16871-21apt-get20:41:400
396199230,1cyclictest0-21swapper/000:16:300
396199230,0cyclictest31254-21diskmemload23:32:210
396199230,0cyclictest0-21swapper/023:48:230
396199230,0cyclictest0-21swapper/021:35:180
3971992220,2cyclictest2392-21sh22:10:453
3971992219,2cyclictest4137-21munin-run00:35:013
397199220,2cyclictest16999-21ssh21:26:333
397199220,22cyclictest32593-21ssh21:11:223
397199220,22cyclictest0-21swapper/323:08:033
397199220,22cyclictest0-21swapper/300:19:533
397199220,21cyclictest17877-21ssh23:21:153
397199220,21cyclictest16438-21perf19:35:003
397199220,21cyclictest12060-21apt-get20:31:423
397199220,21cyclictest0-21swapper/322:50:033
397199220,21cyclictest0-21swapper/320:51:413
397199220,21cyclictest0-21swapper/320:36:413
397199220,21cyclictest0-21swapper/300:10:573
397199220,1cyclictest8738-21sh21:18:143
397199220,1cyclictest4462-21ssh22:41:203
397199220,1cyclictest25620-21ssh21:33:173
397199220,0cyclictest31254-21diskmemload00:25:343
3966992222,0cyclictest31254-21diskmemload22:25:062
3966992222,0cyclictest31254-21diskmemload00:10:512
3966992222,0cyclictest0-21swapper/223:25:082
3966992222,0cyclictest0-21swapper/222:16:052
3966992222,0cyclictest0-21swapper/222:00:212
3966992222,0cyclictest0-21swapper/221:55:382
3966992222,0cyclictest0-21swapper/221:35:182
396699220,2cyclictest19939-21sh00:20:062
396699220,22cyclictest0-21swapper/223:20:472
396699220,21cyclictest3533-21ssh00:05:472
396699220,21cyclictest28859-21cron21:05:002
396699220,21cyclictest1749-21ssh23:35:052
396699220,1cyclictest2475-21ssh00:32:472
396699220,1cyclictest17767-21sh00:17:262
3963992222,0cyclictest0-21swapper/122:41:111
3963992222,0cyclictest0-21swapper/121:51:021
3963992221,1cyclictest0-21swapper/122:10:331
3963992220,1cyclictest0-21swapper/122:31:181
396399220,21cyclictest8958-21ssh23:12:171
396399220,21cyclictest32015-21apt-get20:06:421
396399220,21cyclictest0-21swapper/121:25:581
396399220,21cyclictest0-21swapper/121:15:211
396399220,21cyclictest0-21swapper/121:06:401
396399220,1cyclictest0-21swapper/123:52:121
396399220,1cyclictest0-21swapper/122:36:461
396399220,1cyclictest0-21swapper/100:26:141
396399220,0cyclictest31254-21diskmemload21:47:241
3961992221,1cyclictest0-21swapper/022:32:340
3961992220,2cyclictest18508-21sh23:50:010
3961992220,1cyclictest16996-21ssh23:20:090
396199220,22cyclictest0-21swapper/000:21:040
396199220,22cyclictest0-21swapper/000:10:510
396199220,21cyclictest24809-21apt-get19:51:410
396199220,21cyclictest16445-21munin-run19:35:000
3971992121,0cyclictest0-21swapper/323:45:093
3971992121,0cyclictest0-21swapper/322:45:503
3971992119,1cyclictest606-21dbus-daemon20:25:003
3971992119,1cyclictest13635-21apt-get19:30:003
397199210,21cyclictest0-21swapper/322:36:473
397199210,20cyclictest0-21swapper/320:41:403
397199210,1cyclictest4399-21apt-get20:16:413
3966992121,0cyclictest0-21swapper/219:56:402
396699210,20cyclictest0-21swapper/220:16:412
396699210,0cyclictest28941-21ssh23:30:332
396699210,0cyclictest0-21swapper/220:41:402
3963992121,0cyclictest0-21swapper/119:11:411
3963992120,1cyclictest0-21swapper/120:45:021
3961992121,0cyclictest0-21swapper/020:11:410
3961992120,1cyclictest4366-21apt-get19:11:410
3961992120,1cyclictest0-21swapper/022:20:110
3961992119,1cyclictest8863-21sh20:25:000
396199210,21cyclictest0-21swapper/022:25:470
396199210,21cyclictest0-21swapper/020:35:010
396199210,0cyclictest0-21swapper/020:56:410
396199210,0cyclictest0-21swapper/019:56:400
3971992020,0cyclictest0-21swapper/321:06:403
3971992020,0cyclictest0-21swapper/320:06:423
3971992019,1cyclictest0-21swapper/319:11:413
3971992018,1cyclictest18829-21sendmail19:40:003
397199200,0cyclictest0-21swapper/320:26:413
397199200,0cyclictest0-21swapper/320:26:413
3966992020,0cyclictest0-21swapper/219:51:412
3963992020,0cyclictest0-21swapper/120:41:401
3963992020,0cyclictest0-21swapper/120:21:411
3963992019,0cyclictest0-21swapper/119:45:021
3963992018,1cyclictest0-21swapper/120:56:411
396399200,18cyclictest11285-21idleruntime-cro20:30:011
396399200,18cyclictest11285-21idleruntime-cro20:30:011
396399200,0cyclictest0-21swapper/120:16:411
3961992017,2cyclictest18833-21dump-pmu-power19:40:000
396199200,1cyclictest24085-21grep20:55:000
396199200,19cyclictest11249-21apt19:25:000
3971991919,0cyclictest0-21swapper/319:16:423
3966991919,0cyclictest0-21swapper/220:11:412
3966991918,1cyclictest0-21swapper/220:56:412
3966991917,1cyclictest8871-21latency_hist20:25:002
3966991917,1cyclictest0-21swapper/219:25:002
3963991919,0cyclictest0-21swapper/120:01:441
3963991919,0cyclictest0-21swapper/119:41:411
3961991919,0cyclictest0-21swapper/019:40:020
3961991917,1cyclictest0-21swapper/019:45:020
3971991818,0cyclictest0-21swapper/319:21:403
3971991817,1cyclictest21634-21apt-get19:45:023
3971991816,1cyclictest24453-21apt-get20:56:413
3966991818,0cyclictest0-21swapper/219:11:412
3966991817,0cyclictest0-21swapper/220:46:422
3966991816,1cyclictest22254-21apt-get19:45:022
396699180,0cyclictest24064-21perf20:55:002
3963991817,1cyclictest27203-21apt-get19:56:401
396399180,17cyclictest313-21systemd-journal20:40:011
3961991818,0cyclictest0-21swapper/020:46:420
3961991818,0cyclictest0-21swapper/020:36:410
3961991818,0cyclictest0-21swapper/020:26:410
3961991818,0cyclictest0-21swapper/020:26:410
3961991818,0cyclictest0-21swapper/019:16:420
3971991717,0cyclictest0-21swapper/320:45:023
3971991717,0cyclictest0-21swapper/320:01:443
3971991715,1cyclictest0-21swapper/319:45:013
3966991717,0cyclictest0-21swapper/219:40:022
3966991716,1cyclictest9266-21apt-get20:26:412
3966991716,1cyclictest9266-21apt-get20:26:412
3966991716,1cyclictest14455-21apt-get20:36:412
3966991716,0cyclictest0-21swapper/220:31:422
3966991715,1cyclictest18823-21apt-get19:40:002
3963991717,0cyclictest0-21swapper/119:21:411
3963991716,1cyclictest26830-21awk19:55:011
3963991716,0cyclictest6759-21apt-get19:16:421
3963991715,1cyclictest11627-21apt-get19:26:401
396399170,16cyclictest4011-21latency_hist20:15:001
3971991615,1cyclictest0-21swapper/319:56:403
3971991614,1cyclictest4022-21perf20:15:003
3971991614,1cyclictest0-21swapper/319:55:013
3966991616,0cyclictest0-21swapper/219:16:422
3966991615,0cyclictest14424-21apt-get19:31:422
3966991614,1cyclictest31287-21munin-run21:10:002
3966991614,1cyclictest29624-21apt-get20:01:432
396399160,1cyclictest0-21swapper/120:30:021
396199160,2cyclictest5019-21gltestperf20:16:430
397199150,13cyclictest29717-21munin-node21:00:033
396699150,14cyclictest1314-21users20:06:522
3961991515,0cyclictest0-21swapper/020:01:430
396199150,1cyclictest0-21swapper/020:06:520
396699140,14cyclictest0-21swapper/219:25:302
3963991414,0cyclictest0-21swapper/120:50:101
396199140,0cyclictest0-21swapper/019:25:520
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional