You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 08:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue May 26, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168032970,1sleep00-21swapper/023:49:490
318599901,34cyclictest8876-21unixbench-2d22:20:320
3185998731,4cyclictest6873-21latency_hist20:50:040
3185998530,15cyclictest101ktimersoftd/023:35:280
318599806,15cyclictest0-21swapper/021:20:180
257032800,7sleep1318699cyclictest22:40:531
123362800,6sleep3318899cyclictest23:05:073
312922750,2sleep20-21swapper/222:48:352
318599737,26cyclictest0-21swapper/020:40:160
318699725,47cyclictest7950irq/56-dwc_otg_19:35:171
3185997120,27cyclictest11371-21df21:45:190
3185997120,27cyclictest11371-21df21:45:190
318699692,48cyclictest24014-21wc20:05:301
3185996921,24cyclictest0-21swapper/021:40:150
318699686,28cyclictest0-21swapper/123:25:171
318699686,28cyclictest0-21swapper/123:25:171
318599681,13cyclictest0-21swapper/000:10:150
318699673,49cyclictest8050irq/56-dwc_otg_20:00:041
3186996710,41cyclictest0-21swapper/121:55:181
318899665,40cyclictest8050irq/56-dwc_otg_19:30:163
318699662,16cyclictest23540-21ssh22:00:141
318699661,5cyclictest394-21/usr/sbin/munin20:30:261
318699661,4cyclictest8050irq/56-dwc_otg_22:15:301
318699661,4cyclictest8050irq/56-dwc_otg_22:15:301
318699651,5cyclictest21517-21/usr/sbin/munin23:15:261
72502640,2sleep30-21swapper/321:40:203
318699645,43cyclictest8050irq/56-dwc_otg_21:55:031
318699642,8cyclictest5578-21wget00:15:181
318599648,27cyclictest20121-21aten_rbpower_en22:35:160
318699634,39cyclictest7850irq/56-dwc_otg20:35:201
318699633,45cyclictest8050irq/56-dwc_otg_20:50:201
318699623,4cyclictest8050irq/56-dwc_otg_23:10:041
318699623,49cyclictest19990-21rm22:35:091
318699621,4cyclictest22054-21/usr/sbin/munin00:35:191
318899612,8cyclictest0-21swapper/323:20:153
318699611,50cyclictest1015-21chrt21:32:331
318699611,50cyclictest1015-21chrt21:32:331
318699611,4cyclictest9007-21/usr/sbin/munin23:40:321
318699603,42cyclictest0-21swapper/121:17:011
318699602,3cyclictest0-21swapper/123:49:001
318699601,5cyclictest3469-21ssh00:11:561
3185996022,9cyclictest7950irq/56-dwc_otg_23:05:150
318799592,30cyclictest0-21swapper/219:40:132
318699596,38cyclictest8050irq/56-dwc_otg_22:12:151
318699595,44cyclictest8050irq/56-dwc_otg_00:01:091
318699595,40cyclictest7850irq/56-dwc_otg21:13:001
318699593,45cyclictest0-21swapper/121:39:221
318699593,44cyclictest8050irq/56-dwc_otg_23:55:271
318699593,44cyclictest8050irq/56-dwc_otg_23:55:271
318699592,45cyclictest0-21swapper/121:43:501
318699591,44cyclictest0-21swapper/123:35:261
318699591,42cyclictest9921-21grep20:55:321
318599597,49cyclictest0-21swapper/021:40:030
318599597,32cyclictest0-21swapper/020:25:190
318699584,35cyclictest28365-21missed_timers22:05:251
318699582,41cyclictest0-21swapper/122:55:171
318699581,6cyclictest32375-21ssh00:08:041
318699581,5cyclictest19115-21ssh23:12:501
318699581,5cyclictest15850-21grep19:45:041
318699581,45cyclictest0-21swapper/123:51:211
318599581,44cyclictest9282-21ls20:55:240
318699576,40cyclictest13309-21ssh21:47:061
318699576,40cyclictest13309-21ssh21:47:061
318699574,32cyclictest0-21swapper/122:50:251
318699572,44cyclictest0-21swapper/122:26:561
318699572,2cyclictest0-21swapper/123:04:561
318699571,6cyclictest16922-21taskset00:28:501
3186995715,39cyclictest19311-21latency_hist19:55:041
318699571,4cyclictest0-21swapper/122:20:491
318699571,4cyclictest0-21swapper/119:13:241
318699571,45cyclictest12773-21sh00:23:491
3185995741,3cyclictest0-21swapper/022:08:440
3185995737,5cyclictest7950irq/56-dwc_otg_21:15:190
318599573,44cyclictest20636-21kworker/0:123:55:180
318599573,44cyclictest20636-21kworker/0:123:55:180
318699566,40cyclictest8050irq/56-dwc_otg_00:32:181
318699562,40cyclictest0-21swapper/120:14:081
318699561,4cyclictest0-21swapper/119:25:221
318699561,39cyclictest8050irq/56-dwc_otg_23:32:491
318699561,39cyclictest0-21swapper/122:46:561
318599564,22cyclictest0-21swapper/023:25:170
318599564,22cyclictest0-21swapper/023:25:170
3185995629,13cyclictest7950irq/56-dwc_otg_21:50:330
318699552,5cyclictest30301-21ssh21:29:201
318699551,4cyclictest0-21swapper/120:47:091
318599559,29cyclictest483-21cron21:30:040
318699545,41cyclictest8050irq/56-dwc_otg_20:00:241
318699541,47cyclictest12171-21latency_hist19:35:041
318699541,38cyclictest25997-21ssh21:24:041
318599548,26cyclictest13631-21latency_hist00:25:050
318599547,23cyclictest0-21swapper/023:50:140
318599542,32cyclictest0-21swapper/022:10:320
3185995415,22cyclictest9550irq/80-mmc021:05:040
3185995415,22cyclictest9550irq/80-mmc021:05:040
318899531,6cyclictest0-21swapper/321:35:163
318699533,3cyclictest0-21swapper/121:09:511
318699532,4cyclictest0-21swapper/119:45:171
318699531,5cyclictest24912-21munin-node23:20:141
318699531,4cyclictest8050irq/56-dwc_otg_22:32:391
318699531,46cyclictest0-21swapper/120:44:451
3185995338,9cyclictest7043-21kworker/0:100:35:040
3185995312,18cyclictest0-21swapper/000:30:050
33342520,2sleep20-21swapper/221:35:242
318799523,7cyclictest16549-21ssh23:10:152
318699522,47cyclictest7088-21fschecks_count19:20:201
318699521,4cyclictest28583-21apt-get20:20:161
318699521,38cyclictest28062-21chrt20:16:571
318599524,25cyclictest19430-21ssh21:55:160
3185995225,14cyclictest7850irq/56-dwc_otg23:00:030
318599521,30cyclictest141-21systemd-journal00:03:000
241932520,2sleep30-21swapper/320:05:323
136725213,22sleep31444-21aten_rbpower_po19:05:173
318699512,44cyclictest0-21swapper/119:15:301
318599512,28cyclictest24734-21seq20:10:040
318599512,19cyclictest0-21swapper/021:05:220
318599511,8cyclictest27865-21unixbench_multi20:15:330
318599511,7cyclictest1023-21munin-run00:10:030
318599511,29cyclictest23651-21aten_rbpower_po22:00:170
318599510,48cyclictest22782-21kworker/0:120:05:050
318599505,21cyclictest13-21rcuc/019:10:140
3185995025,11cyclictest15957-21chrt19:45:160
318599501,39cyclictest0-21swapper/023:30:230
318599501,31cyclictest0-21swapper/020:55:040
318599501,28cyclictest2411-21modprobe21:35:040
318599501,28cyclictest2411-21modprobe21:35:040
318599501,26cyclictest26112-21unixbench-2d23:20:320
318599501,25cyclictest21816-21ssh23:15:310
318599501,23cyclictest3819-21rm22:55:030
318699492,5cyclictest12185-21latency_hist21:05:041
318699492,5cyclictest12185-21latency_hist21:05:041
318599497,40cyclictest15636-21cron22:30:030
318599496,34cyclictest0-21swapper/019:35:160
3185994923,6cyclictest141-21systemd-journal19:20:030
3185994920,13cyclictest17250-21ssh22:30:330
3185994916,14cyclictest6110-21sh22:17:490
3185994916,14cyclictest6110-21sh22:17:490
318599491,43cyclictest14260-21aten_rbpower_vo19:40:170
318599491,34cyclictest0-21swapper/000:19:430
318599491,30cyclictest28003-21munin-run22:45:040
318899485,29cyclictest18815-1kworker/3:0H23:15:113
318599486,22cyclictest9-21ksoftirqd/023:40:230
3185994828,8cyclictest8050irq/56-dwc_otg_20:40:030
3185994822,7cyclictest9486-21grep19:25:300
3185994822,3cyclictest0-21swapper/000:35:320
3185994818,6cyclictest26533-21idleruntime-cro20:15:040
318799473,17cyclictest1165-21snmpd20:00:162
318699471,7cyclictest8050irq/56-dwc_otg_20:25:171
3185994618,5cyclictest16521-21/usr/sbin/munin23:10:220
3185994614,15cyclictest18960-21seq19:50:440
318599452,33cyclictest0-21swapper/021:13:010
3185994516,13cyclictest0-21swapper/019:21:080
3185994511,19cyclictest13954-21diskmemload22:49:050
318599442,23cyclictest0-21swapper/023:01:250
25692442,35sleep00-21swapper/019:05:420
318899437,24cyclictest0-21swapper/322:20:183
318899437,22cyclictest0-21swapper/319:50:173
318899432,5cyclictest0-21swapper/322:40:163
318799437,22cyclictest0-21swapper/220:05:172
318799437,21cyclictest0-21swapper/220:55:172
3185994326,11cyclictest521-21in:imuxsock20:21:090
318599431,26cyclictest380-21ntpq20:30:250
190992430,5sleep3318899cyclictest21:15:243
176742430,2sleep20-21swapper/200:30:042
318899428,18cyclictest7850irq/56-dwc_otg00:10:153
318899427,22cyclictest0-21swapper/319:40:163
318899426,5cyclictest8592-21cut23:00:213
3188994217,10cyclictest8050irq/56-dwc_otg_22:00:313
3188994213,14cyclictest8050irq/56-dwc_otg_23:55:233
3188994213,14cyclictest8050irq/56-dwc_otg_23:55:233
318899421,16cyclictest39-21ksoftirqd/323:47:093
318899421,11cyclictest8050irq/56-dwc_otg_23:30:043
318899421,11cyclictest8050irq/56-dwc_otg_23:30:043
318799429,8cyclictest11560-21latency_hist22:25:042
318799427,25cyclictest0-21swapper/219:35:162
3185994222,3cyclictest0-21swapper/019:30:270
3188994119,11cyclictest381ktimersoftd/322:15:183
3188994119,11cyclictest381ktimersoftd/322:15:183
318599412,22cyclictest0-21swapper/019:55:310
318899408,13cyclictest39-21ksoftirqd/320:30:203
318899407,24cyclictest0-21swapper/321:00:173
318899407,24cyclictest0-21swapper/321:00:173
318899407,23cyclictest0-21swapper/319:35:093
318899402,7cyclictest8050irq/56-dwc_otg_21:05:153
3188994013,20cyclictest16566-21ssh23:10:163
318799407,23cyclictest0-21swapper/222:55:162
318799402,7cyclictest0-21swapper/219:15:162
3187994017,7cyclictest3644-21aten_rbpower_vo22:15:172
3187994017,7cyclictest3644-21aten_rbpower_vo22:15:172
318899394,21cyclictest60-21kswapd000:15:153
318899393,23cyclictest0-21swapper/319:15:183
3188993916,13cyclictest16133-21sh21:11:253
3188993913,10cyclictest381ktimersoftd/320:55:283
70862380,1sleep20-21swapper/200:15:592
318899387,7cyclictest8050irq/56-dwc_otg_20:50:163
318899387,13cyclictest381ktimersoftd/300:35:033
318899385,8cyclictest39-21ksoftirqd/322:40:043
318899385,24cyclictest15997-21ssh22:30:173
318899382,5cyclictest973150ssh00:20:183
3188993815,5cyclictest23320-21latency_hist22:00:043
318899377,23cyclictest0-21swapper/320:45:173
318899376,10cyclictest5188-21grep23:35:203
318899374,21cyclictest7850irq/56-dwc_otg00:29:253
3188993718,5cyclictest8050irq/56-dwc_otg_20:00:213
318899371,27cyclictest7178-21ls19:20:213
318799375,9cyclictest8050irq/56-dwc_otg_23:55:052
318799375,6cyclictest0-21swapper/221:50:052
318799375,6cyclictest0-21swapper/221:50:052
3187993722,4cyclictest1552-21/usr/sbin/munin23:30:272
3187993718,5cyclictest2238-21tr20:35:272
3187993713,7cyclictest26552-21cron21:25:042
318899364,12cyclictest39-21ksoftirqd/300:37:493
318899363,20cyclictest381ktimersoftd/319:13:243
318899361,9cyclictest39-21ksoftirqd/321:49:083
318899361,9cyclictest39-21ksoftirqd/321:49:083
3188993618,11cyclictest31332-21sh22:48:393
3188993615,7cyclictest0-21swapper/300:05:253
3188993614,8cyclictest7850irq/56-dwc_otg23:44:203
318899361,20cyclictest21300-21ls20:00:043
3187993613,9cyclictest13954-21diskmemload23:23:592
318799361,25cyclictest19342-21ls19:55:042
318899358,12cyclictest381ktimersoftd/322:09:153
318899357,15cyclictest0-21swapper/320:35:173
318899357,15cyclictest0-21swapper/320:10:173
318899357,14cyclictest381ktimersoftd/322:29:213
3188993514,10cyclictest8050irq/56-dwc_otg_00:01:543
3188993513,9cyclictest381ktimersoftd/322:12:103
3188993513,15cyclictest7950irq/56-dwc_otg_22:55:213
3188993511,14cyclictest381ktimersoftd/323:52:263
318799357,13cyclictest8050irq/56-dwc_otg_22:00:242
318799356,16cyclictest32647-1kworker/u9:123:35:412
318799352,21cyclictest0-21swapper/200:10:132
3187993516,4cyclictest29163-21aten_rbpower_po23:25:182
3187993516,4cyclictest29163-21aten_rbpower_po23:25:182
318799351,5cyclictest13954-21diskmemload22:35:152
318899347,15cyclictest381ktimersoftd/319:30:043
318899347,15cyclictest0-21swapper/319:45:163
318899343,6cyclictest8050irq/56-dwc_otg_21:25:173
318899341,7cyclictest27085-21grep20:15:223
318899341,12cyclictest37-21rcuc/323:33:023
318799348,13cyclictest8050irq/56-dwc_otg_21:10:202
318799347,6cyclictest7850irq/56-dwc_otg00:00:042
318799347,6cyclictest7850irq/56-dwc_otg00:00:042
318799345,4cyclictest7850irq/56-dwc_otg22:10:322
318799344,19cyclictest19651-21aten_rbpower_vo19:55:182
318799341,22cyclictest24213-21ssh00:37:032
318799341,19cyclictest0-21swapper/221:50:242
3187993410,7cyclictest7950irq/56-dwc_otg_00:30:172
318899336,13cyclictest8050irq/56-dwc_otg_20:20:213
318899334,6cyclictest7850irq/56-dwc_otg21:51:283
318799338,5cyclictest7950irq/56-dwc_otg_00:00:152
318799338,10cyclictest0-21swapper/222:31:292
318799334,4cyclictest7850irq/56-dwc_otg23:45:252
318799333,15cyclictest29805-21basename00:05:192
318799332,6cyclictest0-21swapper/220:40:302
318799332,19cyclictest0-21swapper/221:34:062
318799332,19cyclictest0-21swapper/221:34:062
318799331,6cyclictest1910350ssh21:15:242
3187993313,13cyclictest301ktimersoftd/223:02:502
318799331,25cyclictest32013-21ls20:30:042
318799331,22cyclictest604-21switchtime20:30:292
318899323,15cyclictest7950irq/56-dwc_otg_22:50:263
318899321,7cyclictest24512-21seq21:21:213
318799327,6cyclictest11576-21unixbench_singl19:30:312
318799325,5cyclictest7850irq/56-dwc_otg23:40:142
318799324,4cyclictest7950irq/56-dwc_otg_19:20:292
318799324,14cyclictest7950irq/56-dwc_otg_23:06:512
318799324,14cyclictest7950irq/56-dwc_otg_22:26:122
318799322,14cyclictest7950irq/56-dwc_otg_19:15:032
3187993212,14cyclictest0-21swapper/222:09:212
318799321,15cyclictest8050irq/56-dwc_otg_20:45:322
3187993211,4cyclictest0-21swapper/222:40:322
20632327,12sleep10-21swapper/119:05:271
318899318,17cyclictest7950irq/56-dwc_otg_20:40:163
318899312,25cyclictest8050irq/56-dwc_otg_21:30:153
318899312,25cyclictest8050irq/56-dwc_otg_21:30:153
318899312,14cyclictest7950irq/56-dwc_otg_20:25:233
318799319,4cyclictest0-21swapper/221:41:582
318799315,23cyclictest19451-21ssh21:55:192
318799312,20cyclictest21403-21grep23:15:242
13662312,22sleep20-21swapper/219:05:162
318799308,4cyclictest0-21swapper/221:26:042
318799307,12cyclictest8050irq/56-dwc_otg_19:45:232
318799306,5cyclictest8050irq/56-dwc_otg_22:50:232
318799305,15cyclictest0-21swapper/220:10:302
318799302,19cyclictest0-21swapper/200:21:462
3187993019,4cyclictest279632chrt20:15:412
318799295,6cyclictest7850irq/56-dwc_otg21:05:242
3187992914,6cyclictest8919-21fschecks_time19:25:222
111752290,1sleep20-21swapper/221:00:252
111752290,1sleep20-21swapper/221:00:252
318799282,16cyclictest0-21swapper/220:50:102
318799263,4cyclictest28642-21tr20:20:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional