You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-11 - 06:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Mar 11, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24595992928,1cyclictest34-21ksoftirqd/321:48:063
2459599291,0cyclictest34-21ksoftirqd/321:19:553
2459599290,28cyclictest34-21ksoftirqd/321:31:373
24595992621,3cyclictest19409-21diskmemload23:37:093
2459599261,24cyclictest34-21ksoftirqd/322:21:233
2458799260,2cyclictest20455-21ssh00:15:431
2458499260,2cyclictest27145-21sh22:18:240
2459599250,25cyclictest0-21swapper/323:40:563
2459599250,24cyclictest2702-21basename21:25:013
2459599250,24cyclictest19409-21diskmemload22:31:103
2459599250,1cyclictest19409-21diskmemload21:44:443
24591992523,1cyclictest0-21swapper/222:07:042
24591992522,2cyclictest27133-21rm21:47:182
2459199250,1cyclictest0-21swapper/223:06:442
24587992523,2cyclictest20626-21ssh21:11:251
2458799250,24cyclictest19409-21diskmemload23:08:581
2458799250,24cyclictest0-21swapper/122:04:061
2458499250,24cyclictest0-21swapper/000:21:470
2458499250,24cyclictest0-21swapper/000:21:470
24595992423,1cyclictest34-21ksoftirqd/321:27:413
24595992423,0cyclictest0-21swapper/323:51:423
24595992423,0cyclictest0-21swapper/300:06:473
2459599241,22cyclictest34-21ksoftirqd/319:46:403
2459599240,24cyclictest34-21ksoftirqd/300:00:143
2459599240,23cyclictest19409-21diskmemload23:48:093
2459599240,23cyclictest0-21swapper/300:00:003
2459599240,1cyclictest19409-21diskmemload23:04:233
24591992422,2cyclictest11204-21ssh21:31:582
24591992420,4cyclictest0-21swapper/222:42:432
24591992420,4cyclictest0-21swapper/222:31:582
2459199241,22cyclictest15376-21runrttasks22:38:092
2459199240,24cyclictest0-21swapper/221:55:412
2459199240,23cyclictest29558-21apt-get21:50:022
2459199240,23cyclictest19409-21diskmemload22:01:042
2459199240,23cyclictest19409-21diskmemload00:19:312
2459199240,23cyclictest0-21swapper/223:50:582
24587992422,1cyclictest23674-21ssh23:46:591
2458799240,24cyclictest0-21swapper/123:01:061
2458799240,23cyclictest19409-21diskmemload22:48:491
2458799240,23cyclictest19409-21diskmemload22:21:001
2458799240,23cyclictest12227-21ssh00:38:191
2458799240,23cyclictest0-21swapper/121:34:341
2458799240,1cyclictest3632-21ssh22:26:481
2458799240,1cyclictest0-21swapper/121:56:061
2458799240,0cyclictest19409-21diskmemload22:15:571
2458799240,0cyclictest0-21swapper/122:35:591
24584992423,1cyclictest22312-21/usr/sbin/munin21:11:480
24584992423,1cyclictest0-21swapper/022:49:300
24584992421,2cyclictest12829-21ssh23:37:070
2458499241,22cyclictest25352-21cron20:15:010
2458499240,2cyclictest3475-21rm22:57:340
2458499240,24cyclictest19409-21diskmemload23:27:510
2458499240,24cyclictest0-21swapper/022:42:360
2458499240,24cyclictest0-21swapper/022:32:240
2458499240,24cyclictest0-21swapper/022:13:410
2458499240,24cyclictest0-21swapper/021:55:130
2458499240,23cyclictest19776-21apt-get20:05:020
2458499240,23cyclictest0-21swapper/022:02:390
24595992323,0cyclictest34-21ksoftirqd/321:50:483
24595992323,0cyclictest19409-21diskmemload22:43:483
24595992323,0cyclictest19409-21diskmemload00:20:453
24595992323,0cyclictest19409-21diskmemload00:20:453
24595992323,0cyclictest0-21swapper/323:28:513
24595992323,0cyclictest0-21swapper/322:51:053
24595992323,0cyclictest0-21swapper/322:25:183
24595992323,0cyclictest0-21swapper/322:00:413
24595992322,1cyclictest34-21ksoftirqd/323:32:123
24595992321,1cyclictest0-21swapper/300:27:413
2459599230,2cyclictest27805-21munin-run20:20:013
2459599230,23cyclictest0-21swapper/323:12:143
2459599230,23cyclictest0-21swapper/322:57:043
2459599230,23cyclictest0-21swapper/300:10:173
2459599230,22cyclictest27938-21rm22:19:423
2459599230,22cyclictest13297-21ssh22:36:423
2459599230,22cyclictest0-21swapper/323:16:413
24591992323,0cyclictest19409-21diskmemload22:58:392
24591992323,0cyclictest19409-21diskmemload00:20:152
24591992323,0cyclictest19409-21diskmemload00:20:152
24591992323,0cyclictest0-21swapper/223:11:082
24591992323,0cyclictest0-21swapper/222:50:542
24591992323,0cyclictest0-21swapper/221:22:332
24591992323,0cyclictest0-21swapper/200:30:422
24591992321,1cyclictest0-21swapper/221:10:332
24591992320,2cyclictest7969-21ssh21:29:592
24591992320,2cyclictest31678-21ssh22:22:262
24591992320,2cyclictest3068-21munin-run00:30:012
2459199230,2cyclictest8819-21ssh23:02:452
2459199230,23cyclictest0-21swapper/223:55:582
2459199230,22cyclictest31365-21sh23:25:042
2459199230,22cyclictest27018-21sh21:16:452
2459199230,22cyclictest22057-21ssh21:42:262
2459199230,22cyclictest21190-21ssh23:15:192
2459199230,22cyclictest11138-21ssh00:06:462
2459199230,22cyclictest10209-21ssh23:35:402
2459199230,0cyclictest19409-21diskmemload00:39:142
2459199230,0cyclictest0-21swapper/223:40:122
2459199230,0cyclictest0-21swapper/222:18:482
24587992323,0cyclictest0-21swapper/123:39:471
24587992323,0cyclictest0-21swapper/123:29:221
24587992323,0cyclictest0-21swapper/123:12:241
24587992323,0cyclictest0-21swapper/122:43:041
24587992323,0cyclictest0-21swapper/122:08:161
24587992323,0cyclictest0-21swapper/121:20:551
24587992321,1cyclictest27799-21grep20:20:011
24587992321,1cyclictest0-21swapper/100:31:471
24587992321,1cyclictest0-21swapper/100:05:161
24587992320,2cyclictest28769-21sh21:18:081
2458799231,21cyclictest31748-21awk19:25:011
2458799230,2cyclictest31289-21ssh00:26:221
2458799230,2cyclictest17694-21ssh00:12:291
2458799230,23cyclictest29441-21strings22:51:451
2458799230,23cyclictest0-21swapper/121:26:071
2458799230,22cyclictest26147-21rm23:50:241
2458799230,22cyclictest21416-21ssh23:15:401
2458799230,22cyclictest19409-21diskmemload00:00:461
2458799230,22cyclictest17409-21idleruntime-cro20:00:001
2458799230,22cyclictest14160-21ssh21:36:011
2458799230,22cyclictest0-21swapper/121:50:371
2458799230,1cyclictest18335-21ssh22:10:341
2458799230,0cyclictest0-21swapper/123:32:561
24584992323,0cyclictest19409-21diskmemload23:11:310
24584992323,0cyclictest0-21swapper/023:54:120
24584992323,0cyclictest0-21swapper/023:15:010
24584992323,0cyclictest0-21swapper/021:50:200
24584992323,0cyclictest0-21swapper/000:10:280
24584992322,1cyclictest19409-21diskmemload21:25:180
24584992322,1cyclictest16701-21ssh22:08:230
24584992321,1cyclictest0-21swapper/023:55:540
24584992321,1cyclictest0-21swapper/023:08:520
24584992321,1cyclictest0-21swapper/022:36:000
24584992321,1cyclictest0-21swapper/021:37:500
24584992321,1cyclictest0-21swapper/019:50:010
24584992321,1cyclictest0-21swapper/000:15:010
24584992321,1cyclictest0-21swapper/000:07:440
24584992320,2cyclictest32569-21cat00:26:470
24584992320,2cyclictest1655-21ssh22:25:460
2458499230,23cyclictest0-21swapper/021:44:310
2458499230,22cyclictest8829-21sh23:33:580
2458499230,22cyclictest0-21swapper/021:49:120
2458499230,21cyclictest0-21swapper/000:34:190
2458499230,1cyclictest19409-21diskmemload23:00:420
2458499230,1cyclictest0-21swapper/000:00:220
2458499230,0cyclictest0-21swapper/023:49:210
2458499230,0cyclictest0-21swapper/022:54:110
24595992222,0cyclictest0-21swapper/322:45:483
24595992222,0cyclictest0-21swapper/322:14:063
24595992220,1cyclictest3662-21ssh00:30:353
24595992220,1cyclictest20320-21ssh00:15:313
24595992220,1cyclictest0-21swapper/321:10:013
24595992219,2cyclictest29376-21basename19:20:003
2459599220,21cyclictest19409-21diskmemload22:05:303
2459599220,21cyclictest19409-21diskmemload21:55:333
2459599220,21cyclictest17402-21perf20:00:003
2459599220,21cyclictest0-21swapper/323:21:403
2459599220,21cyclictest0-21swapper/320:06:423
2459599220,21cyclictest0-21swapper/319:31:413
2459599220,1cyclictest0-21swapper/321:36:163
2459599220,0cyclictest0-21swapper/323:05:483
24591992222,0cyclictest0-21swapper/223:45:472
24591992222,0cyclictest0-21swapper/222:25:392
24591992221,1cyclictest18572-21diskmemload22:10:562
2459199220,22cyclictest0-21swapper/223:33:512
2459199220,22cyclictest0-21swapper/223:21:032
2459199220,21cyclictest19776-21apt-get20:05:002
2459199220,21cyclictest19409-21diskmemload22:45:302
2459199220,21cyclictest19191-21ssh00:14:492
2459199220,21cyclictest0-21swapper/221:35:222
2459199220,1cyclictest0-21swapper/220:15:012
2459199220,1cyclictest0-21swapper/200:00:092
24587992222,0cyclictest0-21swapper/123:42:511
24587992221,1cyclictest24490-21ssh21:45:591
24587992221,1cyclictest0-21swapper/122:58:221
24587992220,1cyclictest6810-21ssh22:30:341
24587992220,1cyclictest12150-21perf19:50:011
2458799220,22cyclictest0-21swapper/121:41:591
2458799220,21cyclictest25392-21ssh00:20:181
2458799220,21cyclictest25392-21ssh00:20:181
2458799220,21cyclictest0-21swapper/123:21:291
2458799220,1cyclictest29373-21awk19:20:001
2458799220,1cyclictest0-21swapper/123:56:271
24584992221,1cyclictest0-21swapper/021:21:550
24584992220,1cyclictest0-21swapper/023:20:400
2458499220,21cyclictest32578-21latency_hist20:30:010
2458499220,21cyclictest19409-21diskmemload21:16:040
2458499220,21cyclictest0-21swapper/022:20:120
2458499220,21cyclictest0-21swapper/020:56:410
2458499220,1cyclictest0-21swapper/021:30:420
2458499220,0cyclictest0-21swapper/023:40:100
2459599210,20cyclictest0-21swapper/320:46:413
2459599210,20cyclictest0-21swapper/300:35:273
2459599210,0cyclictest0-21swapper/320:36:403
24591992121,0cyclictest7764-21apt-get20:46:412
24591992121,0cyclictest0-21swapper/220:16:392
24591992120,0cyclictest24932-21apt-get19:11:412
2459199210,20cyclictest5374-21apt-get20:41:422
2459199210,20cyclictest0-21swapper/221:06:422
2459199210,20cyclictest0-21swapper/219:56:412
2459199210,1cyclictest22593-21ls20:10:002
24587992121,0cyclictest15025-21apt-get21:01:411
24587992119,1cyclictest26936-21basename19:15:001
24587992117,3cyclictest6940-21perf19:40:011
2458799210,0cyclictest1733-21apt-get19:30:021
2458799210,0cyclictest0-21swapper/120:46:411
2458499210,0cyclictest0-21swapper/019:31:410
2458499210,0cyclictest0-21swapper/019:21:400
24595992020,0cyclictest0-21swapper/320:01:413
2459599200,0cyclictest0-21swapper/321:01:413
24591992020,0cyclictest29745-21apt-get19:21:402
24591992018,1cyclictest1733-21apt-get19:30:012
2459199200,20cyclictest0-21swapper/220:26:422
2459199200,0cyclictest12555-21apt-get20:56:412
2459199200,0cyclictest0-21swapper/221:01:412
24587992020,0cyclictest32141-21apt-get19:26:421
24587992020,0cyclictest2953-21apt-get20:36:401
24587992020,0cyclictest20571-21apt-get20:06:421
24584992020,0cyclictest0-21swapper/020:25:010
24584992020,0cyclictest0-21swapper/019:16:400
24584992019,1cyclictest0-21swapper/019:26:420
2458499200,20cyclictest0-21swapper/020:16:390
24595991919,0cyclictest0-21swapper/320:56:413
24595991919,0cyclictest0-21swapper/319:36:403
24595991919,0cyclictest0-21swapper/319:26:423
24595991917,1cyclictest0-21swapper/319:40:023
24591991919,0cyclictest0-21swapper/219:36:402
24591991918,0cyclictest10158-21apt-get20:51:392
24587991917,1cyclictest6939-21apt-get19:40:021
24584991919,0cyclictest17420-21apt-get21:06:420
24584991919,0cyclictest0-21swapper/021:01:410
24591991817,0cyclictest7732-21apt-get19:41:412
24587991818,0cyclictest28181-21apt-get20:21:421
24587991818,0cyclictest0-21swapper/120:41:421
24587991818,0cyclictest0-21swapper/119:51:421
24584991818,0cyclictest0-21swapper/020:51:400
24595991717,0cyclictest0-21swapper/319:11:413
2459199170,0cyclictest0-21swapper/220:31:412
24584991717,0cyclictest0-21swapper/020:41:420
24584991717,0cyclictest0-21swapper/020:36:400
24584991715,1cyclictest0-21swapper/019:40:010
24595991616,0cyclictest0-21swapper/319:21:403
24595991612,3cyclictest13354-21awk19:51:433
24591991616,0cyclictest0-21swapper/220:21:422
24591991616,0cyclictest0-21swapper/219:31:412
24591991616,0cyclictest0-21swapper/219:16:402
24591991615,0cyclictest10125-21apt-get19:46:402
24587991616,0cyclictest0-21swapper/120:51:391
24587991615,1cyclictest19441-21munin-plugin-st21:10:011
24587991614,1cyclictest25363-21cron20:15:011
2458799160,15cyclictest30574-21apt-get20:26:411
24584991614,1cyclictest7679-21taskset19:41:240
2458499160,15cyclictest0-21swapper/019:56:450
2459599153,6cyclictest34-21ksoftirqd/320:41:423
24595991515,0cyclictest0-21swapper/320:31:413
24595991512,2cyclictest0-21swapper/320:21:423
2459599150,14cyclictest0-21swapper/320:10:473
2459599150,0cyclictest0-21swapper/320:28:083
2459199151,13cyclictest3378-21diskstats20:36:422
24587991515,0cyclictest17783-21apt-get20:01:411
24584991515,0cyclictest0-21swapper/020:01:410
24584991514,1cyclictest0-21swapper/020:34:230
2458499150,1cyclictest0-21swapper/019:51:470
2459599140,14cyclictest0-21swapper/320:50:153
2459199140,14cyclictest0-21swapper/219:50:292
2458799140,14cyclictest0-21swapper/120:55:291
2458499140,14cyclictest0-21swapper/020:45:450
2458499140,0cyclictest0-21swapper/019:11:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional