You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 08:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon May 11, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2298726000,9sleep122229-21/usr/sbin/munin04:26:221
2225621990,5sleep20-21swapper/204:26:112
1820521090,6sleep32405899cyclictest06:48:483
1820521090,6sleep32405899cyclictest06:48:483
112042960,2sleep00-21swapper/008:35:100
45392790,6sleep22405799cyclictest09:45:502
24055997657,7cyclictest7950irq/56-dwc_otg_09:06:090
288492750,2sleep10-21swapper/107:38:461
44312740,5sleep22405799cyclictest05:06:102
208222740,2sleep20-21swapper/208:06:372
2405599683,32cyclictest0-21swapper/006:31:090
24057996713,12cyclictest7950irq/56-dwc_otg_07:46:092
2405599665,57cyclictest13-21rcuc/006:36:230
2405599656,46cyclictest0-21swapper/009:16:110
24055996535,16cyclictest0-21swapper/007:26:100
2405599651,32cyclictest0-21swapper/007:56:100
2405799635,11cyclictest19795-21ssh06:51:092
24055996215,23cyclictest12957-21latency_hist09:55:570
2405599618,28cyclictest0-21swapper/008:06:080
2405599616,51cyclictest13-21rcuc/007:06:250
2405599616,42cyclictest0-21swapper/004:41:090
24055996136,16cyclictest0-21swapper/008:20:520
2405599605,13cyclictest7850irq/56-dwc_otg_06:11:080
2405599602,38cyclictest0-21swapper/005:45:570
24055995911,32cyclictest17313-21grep08:41:260
24055995820,18cyclictest4666-21/usr/sbin/munin07:11:200
24055995820,18cyclictest4666-21/usr/sbin/munin07:11:200
24055995815,5cyclictest9550irq/80-mmc008:25:570
24055995815,5cyclictest9550irq/80-mmc008:25:570
24057995712,8cyclictest7950irq/56-dwc_otg_06:31:072
2405599573,7cyclictest0-21swapper/005:46:090
2405799568,12cyclictest7750irq/56-dwc_otg08:21:102
2405799568,12cyclictest7750irq/56-dwc_otg08:21:102
2405599569,23cyclictest0-21swapper/006:46:140
2405599569,23cyclictest0-21swapper/006:46:140
2405599565,18cyclictest20877-21ssh09:26:090
24055995642,4cyclictest7850irq/56-dwc_otg_07:31:080
24055995629,12cyclictest7850irq/56-dwc_otg_08:15:570
113442560,3sleep311343-21aten_rbpower_en06:41:113
259472550,2sleep00-21swapper/006:05:580
2405599552,6cyclictest0-21swapper/007:16:080
24055995442,5cyclictest13496-21aten_rbpower_en07:21:110
24055995442,5cyclictest13496-21aten_rbpower_en07:21:110
2405599543,25cyclictest0-21swapper/006:51:090
24055995427,4cyclictest29680-21grep07:01:260
24055995426,6cyclictest9-21ksoftirqd/009:21:180
24055995415,4cyclictest25112-21timerandwakeup06:56:250
2405599538,25cyclictest0-21swapper/008:40:570
2405599533,18cyclictest0-21swapper/007:36:090
24055995313,19cyclictest11611-21/usr/sbin/munin05:26:080
2405599530,46cyclictest18265-21kworker/0:109:32:500
2405799525,8cyclictest7950irq/56-dwc_otg_09:56:092
2405599523,28cyclictest0-21swapper/009:11:110
24055995225,4cyclictest7850irq/56-dwc_otg_08:03:330
2405599522,31cyclictest0-21swapper/009:36:270
24055995130,5cyclictest31785-21grep06:21:140
24055995127,6cyclictest7850irq/56-dwc_otg_05:06:110
24055995125,13cyclictest7750irq/56-dwc_otg04:56:160
2405599511,22cyclictest21437-21unixbench_multi08:46:230
2405599511,22cyclictest21437-21unixbench_multi08:46:230
24055995022,5cyclictest3773-21wget08:26:120
24055995013,4cyclictest2055-21unixbench_multi09:41:250
2405599501,30cyclictest0-21swapper/007:50:370
24055995011,17cyclictest10457-21latency_hist07:55:570
2405799495,5cyclictest7950irq/56-dwc_otg_07:51:132
24056994916,7cyclictest427-21grep08:21:231
24056994916,7cyclictest427-21grep08:21:231
24055994925,5cyclictest29778-21ssh08:56:270
24055994924,4cyclictest10467-21ntp_kernel_err05:21:190
24055994922,3cyclictest0-21swapper/004:36:220
2405599491,4cyclictest6243-21grep09:46:260
2405599491,40cyclictest0-21swapper/006:16:100
2405599491,40cyclictest0-21swapper/006:16:100
231542493,36sleep323289-21grep04:26:253
24058994822,8cyclictest20943-21aten_rbpower_en09:26:113
2405799488,27cyclictest0-21swapper/207:01:032
2405699485,16cyclictest492-21fschecks_count09:01:161
2405599485,4cyclictest13-21rcuc/008:53:110
24055994818,4cyclictest0-21swapper/005:15:570
2405599481,7cyclictest31776-21users07:41:280
190652480,2sleep10-21swapper/106:50:191
190652480,2sleep10-21swapper/106:50:191
2405799477,25cyclictest0-21swapper/206:11:102
2405599473,42cyclictest13-21rcuc/004:50:570
2405599471,7cyclictest520-21cron06:30:560
2405799467,25cyclictest0-21swapper/204:36:102
2405799465,8cyclictest7850irq/56-dwc_otg_08:56:122
2405599461,42cyclictest0-21swapper/006:41:160
2405599461,19cyclictest0-21swapper/009:56:160
2405899457,24cyclictest0-21swapper/304:56:123
2405899456,22cyclictest0-21swapper/305:26:113
2405799457,25cyclictest0-21swapper/205:51:102
24055994520,5cyclictest9-21ksoftirqd/005:55:570
24055994517,14cyclictest31371-21wget04:51:110
24055994517,14cyclictest31371-21wget04:51:110
2405599451,22cyclictest3658-21grep05:01:250
2405799448,19cyclictest29-21rcuc/208:50:542
2405799448,19cyclictest29-21rcuc/208:50:542
2405799446,24cyclictest0-21swapper/204:56:022
2405699447,17cyclictest13700-21fschecks_time05:31:131
2405699447,17cyclictest13700-21fschecks_time05:31:131
2405699446,23cyclictest0-21swapper/105:26:111
24056994415,7cyclictest7064-21aten_rbpower_cu06:36:121
24056994414,14cyclictest24896-21grep08:11:261
24055994412,28cyclictest23557-21unixbench_multi05:56:250
234672442,35sleep00-21swapper/004:26:540
2405799437,25cyclictest0-21swapper/205:36:112
2405799437,22cyclictest0-21swapper/204:41:092
2405599435,35cyclictest9-21ksoftirqd/005:31:350
2405599435,35cyclictest9-21ksoftirqd/005:31:350
24055994317,12cyclictest0-21swapper/009:03:090
31202420,5sleep32405899cyclictest09:43:133
2405899427,24cyclictest0-21swapper/305:21:113
2405699422,10cyclictest5026-21sed08:26:271
24056994217,4cyclictest19074-21latency_hist08:05:571
24056994210,4cyclictest0-21swapper/107:51:251
24055994218,12cyclictest908-21runrttasks05:38:060
24055994213,14cyclictest0-21swapper/006:06:090
2405899413,28cyclictest32486-21ls06:21:243
2405699417,25cyclictest0-21swapper/105:21:111
2405699414,23cyclictest32678-21ssh07:06:021
2405699413,9cyclictest20919-21tr09:26:111
24056994122,4cyclictest6675-21sh06:35:571
24056994117,7cyclictest0-21swapper/106:06:181
2405699411,11cyclictest14506-21grep09:56:261
2405599411,14cyclictest0-21swapper/004:31:140
92992400,1sleep10-21swapper/108:31:411
2405799407,24cyclictest0-21swapper/204:51:112
2405799407,24cyclictest0-21swapper/204:51:112
2405799407,24cyclictest0-21swapper/204:46:122
2405799406,24cyclictest0-21swapper/207:16:112
2405799406,16cyclictest301ktimersoftd/209:04:342
2405799403,12cyclictest31-21ksoftirqd/206:47:462
2405799403,12cyclictest31-21ksoftirqd/206:47:462
24057994016,6cyclictest27599-21sh08:54:552
24057994012,8cyclictest7750irq/56-dwc_otg05:46:092
24056994011,6cyclictest24802-21tr06:56:201
24058993918,6cyclictest2448550wget08:51:113
2405899391,23cyclictest4910-21ls05:06:173
2405699396,10cyclictest7950irq/56-dwc_otg_07:31:121
24056993922,5cyclictest7950irq/56-dwc_otg_09:51:261
2405699392,10cyclictest6181-21awk07:50:571
2405699391,9cyclictest19683-21cut08:06:141
24056993919,5cyclictest12952-21cron07:20:581
24056993919,5cyclictest12952-21cron07:20:581
2405599391,6cyclictest9163-21unixbench_singl05:16:260
2405899388,15cyclictest7850irq/56-dwc_otg_08:36:113
2405899386,21cyclictest4617-21ssh07:11:033
2405899386,21cyclictest4617-21ssh07:11:033
24058993826,6cyclictest14842-21ssh09:18:073
24058993813,11cyclictest23483-21ssh08:49:583
24058993813,11cyclictest23483-21ssh08:49:583
2405899381,26cyclictest29206-21sh08:56:203
2405899381,23cyclictest30814-21ssh07:41:163
2405799382,9cyclictest25379-21rm06:56:272
2405699388,14cyclictest0-21swapper/109:00:391
2405699383,7cyclictest20729-21/usr/sbin/munin06:51:251
2405699383,4cyclictest7850irq/56-dwc_otg_09:31:161
2405699382,7cyclictest29451-21wc04:45:581
2405699381,7cyclictest931950ssh07:16:141
24056993811,14cyclictest0-21swapper/107:56:211
24056993810,4cyclictest0-21swapper/108:41:211
87532370,3sleep324054-21cyclictest08:31:233
24058993724,6cyclictest9351-21ssh09:11:233
2405899371,23cyclictest14048-21missed_timers07:21:193
2405899371,23cyclictest14048-21missed_timers07:21:193
2405799378,17cyclictest29-21rcuc/208:01:082
2405799376,22cyclictest0-21swapper/209:21:112
2405699373,3cyclictest7950irq/56-dwc_otg_05:16:101
2405699372,9cyclictest17092-21grep05:41:101
2405699371,7cyclictest572-21munin-run06:25:561
2405699371,6cyclictest5713-21grep09:46:201
2405699371,25cyclictest19833-21ls05:46:251
177812370,1sleep20-21swapper/208:42:052
118862370,1sleep20-21swapper/208:36:012
292532360,6sleep12405699cyclictest07:01:221
2405899362,24cyclictest0-21swapper/306:51:563
24058993617,10cyclictest13740-21fschecks_time05:31:143
24058993617,10cyclictest13740-21fschecks_time05:31:143
24058993613,5cyclictest2776850wget04:41:103
2405899361,21cyclictest6511-21uniq05:11:133
2405899361,18cyclictest10865-21ssh09:52:163
2405799368,15cyclictest29-21rcuc/207:45:572
2405799368,15cyclictest0-21swapper/208:26:122
2405799368,12cyclictest29-21rcuc/208:11:222
2405799366,14cyclictest7950irq/56-dwc_otg_07:10:582
2405799362,7cyclictest0-21swapper/205:01:102
2405699364,5cyclictest0-21swapper/105:36:091
2405699362,5cyclictest7950irq/56-dwc_otg_09:41:101
24058993523,5cyclictest28897-21ssh07:38:513
24058993510,5cyclictest5013-21grep08:26:273
2405799359,11cyclictest7750irq/56-dwc_otg07:37:032
2405799358,19cyclictest0-21swapper/209:31:142
2405799357,15cyclictest301ktimersoftd/207:34:392
2405799356,4cyclictest7850irq/56-dwc_otg_05:11:132
2405799354,6cyclictest7950irq/56-dwc_otg_06:16:172
2405799354,6cyclictest7950irq/56-dwc_otg_06:16:172
2405799354,16cyclictest7950irq/56-dwc_otg_05:56:242
2405699359,13cyclictest7950irq/56-dwc_otg_05:10:571
24056993515,12cyclictest27900-21basename06:11:101
24056993514,6cyclictest18484-21sh09:22:131
24056993514,13cyclictest20197-21/usr/sbin/munin08:46:051
24056993514,13cyclictest20197-21/usr/sbin/munin08:46:051
24056993514,10cyclictest28009-21sh08:55:391
2405899349,12cyclictest7950irq/56-dwc_otg_06:36:193
2405899349,10cyclictest17928-21unixbench-2d09:21:263
2405899341,6cyclictest356350ssh07:46:263
2405899341,24cyclictest26334-21cut06:06:153
2405799347,8cyclictest7850irq/56-dwc_otg_05:20:572
2405799347,6cyclictest7950irq/56-dwc_otg_08:16:242
2405799347,11cyclictest7750irq/56-dwc_otg05:21:132
2405799345,4cyclictest0-21swapper/207:26:262
2405699347,7cyclictest1055-21fschecks_count06:26:161
2405699344,8cyclictest13802-21grep09:16:251
2405699343,6cyclictest0-21swapper/107:11:151
2405699343,6cyclictest0-21swapper/107:11:151
2405699342,21cyclictest0-21swapper/105:56:511
2405699341,9cyclictest940-21cpuspeed04:56:131
24056993415,5cyclictest29744-21/usr/sbin/munin04:46:151
24056993413,6cyclictest32525-21sh07:42:451
2405699341,23cyclictest29044-21timerwakeupswit08:16:241
24056993410,5cyclictest0-21swapper/109:11:281
2405899336,18cyclictest0-21swapper/306:31:193
24058993313,13cyclictest33232chrt07:09:213
24058993312,5cyclictest20715-21/usr/sbin/munin05:51:113
2405899331,14cyclictest0-21swapper/304:32:183
24058993310,4cyclictest0-21swapper/308:05:583
2405799335,19cyclictest12632-21ssh09:16:112
2405799333,18cyclictest29-21rcuc/206:01:232
2405799332,5cyclictest7850irq/56-dwc_otg_09:06:162
2405799331,5cyclictest31-21ksoftirqd/209:30:142
24057993313,9cyclictest0-21swapper/207:11:102
24057993313,9cyclictest0-21swapper/207:11:102
24057993310,10cyclictest0-21swapper/207:25:442
24057993310,10cyclictest0-21swapper/207:25:442
2405699336,5cyclictest8403-1kworker/1:2H06:16:131
2405699336,5cyclictest8403-1kworker/1:2H06:16:131
2405699334,6cyclictest20964-21fschecks_time05:51:141
2405699333,5cyclictest7950irq/56-dwc_otg_06:01:101
2405699333,14cyclictest7850irq/56-dwc_otg_07:26:131
24056993313,12cyclictest0-21swapper/108:39:191
24056993312,6cyclictest32676-21taskset09:40:321
2405899329,4cyclictest0-21swapper/308:00:093
2405899327,14cyclictest7850irq/56-dwc_otg_07:16:153
2405899326,14cyclictest22032-21ssh07:31:113
2405899325,13cyclictest0-21swapper/308:10:593
2405899324,19cyclictest24254-21apt-get06:01:093
2405899324,13cyclictest7850irq/56-dwc_otg_08:21:123
2405899324,13cyclictest7850irq/56-dwc_otg_08:21:123
2405899323,22cyclictest30008-21ls04:46:183
2405899323,21cyclictest7032-21ssh07:51:173
2405899323,20cyclictest506-21ls09:40:573
2405899321,26cyclictest15576-21idleruntime05:36:153
2405899321,22cyclictest1313-21timerwakeupswit09:01:263
2405799329,10cyclictest7950irq/56-dwc_otg_06:36:552
2405799326,12cyclictest7950irq/56-dwc_otg_09:15:572
2405799323,17cyclictest29-21rcuc/208:31:092
2405799322,5cyclictest8767-21munin-run09:50:572
2405699322,5cyclictest0-21swapper/105:11:131
24056993215,7cyclictest0-21swapper/109:10:141
2405899319,3cyclictest0-21swapper/307:05:553
2405899316,7cyclictest7950irq/56-dwc_otg_09:56:263
2405899316,13cyclictest7950irq/56-dwc_otg_06:11:243
2405899311,9cyclictest0-21swapper/307:26:133
2405799317,14cyclictest0-21swapper/206:06:112
2405799316,3cyclictest0-21swapper/206:41:162
2405799315,14cyclictest301ktimersoftd/209:52:382
2405799311,17cyclictest7950irq/56-dwc_otg_09:36:232
2405799311,14cyclictest31-21ksoftirqd/205:26:172
2405699313,13cyclictest7950irq/56-dwc_otg_04:31:101
24056993118,4cyclictest13867-21sh06:43:391
2405899308,12cyclictest7950irq/56-dwc_otg_09:46:213
2405899307,12cyclictest25080-21aten_rbpower_en09:31:113
2405899304,14cyclictest7950irq/56-dwc_otg_08:18:043
2405899304,11cyclictest7850irq/56-dwc_otg_04:36:143
2405899302,13cyclictest0-21swapper/306:26:493
2405899301,23cyclictest18599-21latency_hist05:45:573
24058993011,5cyclictest7187-21sh09:09:143
2405899301,15cyclictest0-21swapper/305:20:583
2405899301,10cyclictest16117-21ssh08:41:093
24058993010,4cyclictest0-21swapper/306:57:403
2405799302,15cyclictest0-21swapper/207:57:122
2405799301,7cyclictest7850irq/56-dwc_otg_04:35:582
2405799301,18cyclictest17352-21fschecks_time05:41:142
2405699306,5cyclictest31427-21aten_rbpower_vo04:51:111
2405699306,5cyclictest31427-21aten_rbpower_vo04:51:111
2405899296,10cyclictest0-21swapper/308:06:263
24058992925,2cyclictest0-21swapper/306:16:223
24058992925,2cyclictest0-21swapper/306:16:223
2405899291,18cyclictest0-21swapper/304:51:083
2405899291,18cyclictest0-21swapper/304:51:083
2405899291,14cyclictest0-21swapper/305:58:453
2405799297,11cyclictest7950irq/56-dwc_otg_06:21:202
2405699297,5cyclictest0-21swapper/104:36:121
2405699291,6cyclictest2623-21aten_rbpower_cu05:01:101
2405799282,5cyclictest0-21swapper/205:31:092
2405799282,5cyclictest0-21swapper/205:31:092
2405899271,18cyclictest0-21swapper/305:46:563
2405799271,5cyclictest7950irq/56-dwc_otg_06:26:142
2405899266,8cyclictest0-21swapper/305:02:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional