You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 20:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Sun Aug 16, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1966021710,4sleep31161299cyclictest07:30:313
11609991577,88cyclictest0-21swapper/009:15:150
11609991404,110cyclictest0-21swapper/011:20:370
11609991394,109cyclictest12903-21grep10:15:380
11610991362,49cyclictest7950irq/56-dwc_otg_09:15:151
116109912230,56cyclictest2115-21ssh11:20:361
11609991214,8cyclictest5899-21ntp_states11:25:300
116099912136,68cyclictest7850irq/56-dwc_otg_12:30:080
116099912136,68cyclictest7850irq/56-dwc_otg_12:30:080
116099912118,78cyclictest0-21swapper/011:50:220
116099912118,78cyclictest0-21swapper/011:50:220
11609991202,9cyclictest28803-21ls09:55:340
3072521150,5sleep31161299cyclictest09:19:523
11609991142,6cyclictest0-21swapper/009:45:320
11609991082,6cyclictest0-21swapper/011:05:290
116099910431,48cyclictest7850irq/56-dwc_otg_10:25:110
116099910431,48cyclictest7850irq/56-dwc_otg_10:25:110
116109910234,25cyclictest12651-21/usr/sbin/munin10:15:341
1160999963,8cyclictest24600-21fschecks_time07:45:250
11610999327,26cyclictest20045-21ssh10:25:121
11610999327,26cyclictest20045-21ssh10:25:121
11610999219,44cyclictest7950irq/56-dwc_otg_11:25:331
1160999921,9cyclictest10467-21wc08:35:270
11610999042,23cyclictest20254-21ntp_states09:45:331
260402890,5sleep31161299cyclictest09:52:413
226522890,2sleep20-21swapper/207:40:202
1161299893,76cyclictest27873-21grep12:30:343
1161299893,76cyclictest27873-21grep12:30:343
1160999891,72cyclictest7860-21awk08:30:050
1161299883,77cyclictest5922-21/usr/sbin/munin11:25:323
11610998827,36cyclictest0-21swapper/111:50:191
11610998827,36cyclictest0-21swapper/111:50:191
11611998752,21cyclictest7950irq/56-dwc_otg_09:15:152
1161299863,76cyclictest20123-21sort09:45:293
1160999861,64cyclictest21252-21ssh11:45:060
1161299853,63cyclictest0-21swapper/311:20:373
1161199859,58cyclictest0-21swapper/211:25:312
1161299835,64cyclictest21492-21ssh11:05:313
1160999831,68cyclictest0-21swapper/012:20:310
1160999831,68cyclictest0-21swapper/012:20:310
181732820,3sleep318169-21ssh11:40:263
1161299823,72cyclictest12708-21/usr/sbin/munin10:15:343
1161199828,51cyclictest7850irq/56-dwc_otg_09:45:322
1161199823,29cyclictest0-21swapper/211:05:292
11610998226,28cyclictest27925-21ssh12:30:351
11610998226,28cyclictest27925-21ssh12:30:351
1160999811,67cyclictest0-21swapper/007:13:350
11611998028,23cyclictest2086-21/usr/sbin/munin11:20:362
11609998016,46cyclictest7950irq/56-dwc_otg_12:15:120
1160999788,6cyclictest7304-21latency_hist10:10:060
1161299774,67cyclictest28905-21ssh09:55:363
1160999771,6cyclictest1191-21snmpd11:19:240
1161299752,67cyclictest20033-21runrttasks10:25:103
1161299752,67cyclictest20033-21runrttasks10:25:103
1161199753,19cyclictest27792-21grep12:30:332
1161199753,19cyclictest27792-21grep12:30:332
1161099752,37cyclictest21398-21missed_timers11:05:291
1160999752,6cyclictest7850irq/56-dwc_otg_10:30:150
1160999752,52cyclictest15026-21ls07:20:060
11609997516,44cyclictest7892-21ssh09:30:310
11609997514,58cyclictest20505-21cat07:35:060
1160999741,62cyclictest22398-21diskmemload09:35:360
1161299733,64cyclictest25723-21rm11:50:203
1161299733,64cyclictest25723-21rm11:50:203
11609997328,25cyclictest7950irq/56-dwc_otg_09:50:210
1160999721,17cyclictest22002-21ssh11:45:260
11609997210,51cyclictest4862-21sh12:04:160
103182720,3sleep211608-21cyclictest09:34:082
11610997111,39cyclictest28999-21grep09:55:361
1160999715,54cyclictest25992-21ssh12:29:440
11609997116,41cyclictest0-21swapper/009:05:240
1161199704,43cyclictest0-21swapper/210:25:102
1161199704,43cyclictest0-21swapper/210:25:102
1160999709,2cyclictest0-21swapper/009:29:540
11609997010,50cyclictest8550-21/usr/sbin/munin10:10:310
1160999692,53cyclictest0-21swapper/010:47:150
1160999691,57cyclictest20043-21ssh11:04:300
11609996815,43cyclictest0-21swapper/010:20:570
1160999681,52cyclictest0-21swapper/009:43:400
11611996719,26cyclictest12927-21ssh10:15:392
1160999671,50cyclictest0-21swapper/010:55:170
11609996713,42cyclictest0-21swapper/011:56:050
11609996710,46cyclictest3033-21ssh10:04:510
11611996621,22cyclictest25704-21apt-get11:50:202
11611996621,22cyclictest25704-21apt-get11:50:202
11609996616,35cyclictest7950irq/56-dwc_otg_12:40:050
1160999661,20cyclictest583-21ssh10:40:270
1160999657,2cyclictest0-21swapper/007:25:060
1160999657,2cyclictest0-21swapper/007:25:060
1160999651,55cyclictest0-21swapper/009:12:100
1160999651,52cyclictest22478-21apt-get07:40:160
1160999651,45cyclictest0-21swapper/011:33:290
1160999648,48cyclictest1191-21snmpd07:55:520
1160999647,46cyclictest0-21swapper/011:12:320
1160999647,43cyclictest29168-21ssh10:35:310
1160999642,12cyclictest0-21swapper/010:52:510
11609996413,36cyclictest7752-21ssh12:07:000
11609996412,40cyclictest2432-21ssh09:24:580
1160999632,46cyclictest0-21swapper/011:37:110
1160999631,58cyclictest8580-21/usr/sbin/munin08:30:260
1160999631,47cyclictest17103-21aten_rbpower_cu07:25:210
11609996312,38cyclictest0-21swapper/008:01:050
1161199625,27cyclictest7750irq/56-dwc_otg09:55:352
1160999621,49cyclictest0-21swapper/007:35:240
11609996213,38cyclictest0-21swapper/008:57:030
11609996212,40cyclictest12245-21ssh12:12:270
159102610,1sleep20-21swapper/211:37:552
1161299612,42cyclictest7950irq/56-dwc_otg_12:10:063
11610996119,23cyclictest24660-21ls09:50:351
1161299606,5cyclictest7950irq/56-dwc_otg_07:45:303
1160999601,46cyclictest0-21swapper/008:22:210
1161099592,14cyclictest29192-21ssh11:15:191
11610995916,23cyclictest26774-21rm10:32:471
1161299586,36cyclictest0-21swapper/312:00:313
1161299585,4cyclictest7950irq/56-dwc_otg_09:30:333
1161299582,52cyclictest13355-21ssh11:35:163
1160999582,45cyclictest0-21swapper/008:52:010
1161299578,28cyclictest0-21swapper/309:35:413
1161299575,5cyclictest792-1kworker/u9:110:32:563
1161299571,3cyclictest7950irq/56-dwc_otg_12:15:343
11610995719,20cyclictest20405-21seq07:34:591
1160999571,49cyclictest0-21swapper/008:43:360
1160999571,49cyclictest0-21swapper/008:43:360
1160999571,43cyclictest1521-21grep08:10:290
1161299561,50cyclictest18355-21grep08:55:353
1161299561,3cyclictest7950irq/56-dwc_otg_09:25:213
11610995619,12cyclictest13919-21date12:15:061
11610995615,21cyclictest7418-21latency_hist10:10:061
11610995615,11cyclictest21856-21grep11:45:241
1161299554,3cyclictest7950irq/56-dwc_otg_12:40:063
1161299551,5cyclictest7950irq/56-dwc_otg_10:50:273
1161299551,38cyclictest0-21swapper/311:03:143
11610995521,12cyclictest7907-21/usr/sbin/munin10:10:231
11610995512,31cyclictest0-21swapper/112:10:061
1160999559,34cyclictest0-21swapper/008:17:070
1160999551,15cyclictest0-21swapper/008:45:200
11609995510,5cyclictest0-21swapper/009:00:290
1161299543,3cyclictest7950irq/56-dwc_otg_10:39:083
11612995413,38cyclictest16301-21sed11:00:063
1161099546,29cyclictest15364-21ssh09:40:201
11610995420,16cyclictest19445-21ssh10:24:401
11610995418,23cyclictest14991-21ps12:15:321
11610995416,21cyclictest22398-21diskmemload10:49:541
11610995415,30cyclictest3042-21cpuspeed09:25:211
11610995413,16cyclictest7212-21cut09:30:221
1161299531,6cyclictest26072-21/usr/sbin/munin07:50:263
1161299531,5cyclictest6156-21ssh10:08:013
11611995335,13cyclictest301ktimersoftd/210:35:192
11610995319,19cyclictest2748-21taskset10:43:061
11610995319,19cyclictest23573-21unixbench_multi07:40:331
11610995318,19cyclictest19495-21sh11:03:311
11610995316,17cyclictest7750irq/56-dwc_otg11:36:481
1160999532,11cyclictest0-21swapper/008:05:190
1160999532,11cyclictest0-21swapper/008:05:190
110052533,12sleep00-21swapper/007:05:510
1161299524,3cyclictest0-21swapper/311:55:223
1161299522,6cyclictest23036-21ssh12:25:273
1161299522,3cyclictest0-21swapper/310:00:183
1161299521,8cyclictest0-21swapper/312:21:083
1161299521,8cyclictest0-21swapper/312:21:083
1161299521,5cyclictest28746-21apt-get11:15:103
1161299521,5cyclictest16128-21ntp_states09:40:303
11610995222,18cyclictest24734-21/usr/sbin/munin07:45:261
11610995220,11cyclictest28638-21diskstats10:35:231
11610995219,14cyclictest1810-21ssh12:39:371
11610995215,13cyclictest19656-21ssh11:42:131
11610995213,24cyclictest12616-21ssh11:34:221
11610995211,24cyclictest0-21swapper/108:35:291
1161299513,7cyclictest0-21swapper/309:10:173
1161299512,4cyclictest7950irq/56-dwc_otg_07:40:233
1161299511,4cyclictest24931-21apt-get11:10:183
1161299511,3cyclictest7950irq/56-dwc_otg_08:35:343
11612995111,37cyclictest11510-21seq11:32:223
11610995120,11cyclictest1223-21ssh10:01:331
11610995117,20cyclictest21496-21/usr/sbin/munin09:05:281
11610995113,26cyclictest21141-21uname07:35:261
11610995111,14cyclictest31349-21ssh11:55:561
1160999515,12cyclictest0-21swapper/007:53:170
1161299506,4cyclictest8184-21/usr/sbin/munin08:30:203
1161299502,9cyclictest0-21swapper/311:48:433
1161299502,8cyclictest4806-21fschecks_time08:20:273
1161299502,7cyclictest29597-21latency_hist08:00:063
1161299502,7cyclictest2938-21fschecks_count08:15:253
1161299502,4cyclictest0-21swapper/310:45:383
1161299501,9cyclictest0-21swapper/312:14:573
11612995018,30cyclictest3318-21ssh10:44:103
1161299501,6cyclictest7950irq/56-dwc_otg_09:05:303
1161299501,4cyclictest7950irq/56-dwc_otg_10:23:033
11610995018,10cyclictest27215-21unixbench_singl07:50:341
1161299492,5cyclictest32648-21grep08:05:333
1161299492,5cyclictest32648-21grep08:05:333
1161299492,5cyclictest20974-21fschecks_time07:35:243
1161299492,4cyclictest0-21swapper/308:25:243
1161299491,8cyclictest11967-21grep08:40:243
1161299491,8cyclictest11967-21grep08:40:243
11612994917,29cyclictest31387-21sh09:20:213
11612994917,29cyclictest19338-21fschecks_time09:00:243
1161299491,38cyclictest0-21swapper/307:20:063
11610994923,16cyclictest3496-21ssh12:01:451
11610994912,12cyclictest12454-21/usr/sbin/munin10:55:171
11610994910,13cyclictest23968-21ssh12:26:011
11610994910,13cyclictest12999-21ssh09:36:271
1161299482,7cyclictest15186-21ls08:50:053
1161299481,8cyclictest15748-21fschecks_time08:50:243
1161099489,24cyclictest8721-21ls08:30:271
1161099489,17cyclictest24296-21ssh09:10:581
11610994816,10cyclictest807-21ssh09:22:011
11610994815,18cyclictest31517-21systemd-journal10:54:191
11610994814,20cyclictest15134-21sed07:20:061
1161299472,3cyclictest7950irq/56-dwc_otg_07:20:253
1161299472,3cyclictest7950irq/56-dwc_otg_07:20:253
1161299471,8cyclictest18682-21latency_hist07:30:053
1161299471,7cyclictest30778-21grep08:00:333
1161299471,5cyclictest8739-21cat10:10:333
1161099479,12cyclictest31517-21systemd-journal08:48:541
1161099477,14cyclictest7750irq/56-dwc_otg08:19:221
1161099477,11cyclictest0-21swapper/108:11:291
11610994715,11cyclictest28776-21latency_hist11:15:061
11610994714,7cyclictest18788-21munin-run09:00:061
11610994714,19cyclictest1180-21runrttasks09:02:291
11610994713,19cyclictest19580-21/usr/sbin/munin12:20:351
11610994713,19cyclictest19580-21/usr/sbin/munin12:20:351
1161099469,24cyclictest0-21swapper/107:20:071
1161099469,24cyclictest0-21swapper/107:20:071
11610994616,12cyclictest7750irq/56-dwc_otg08:30:051
1161299454,5cyclictest12514-21switchtime07:10:323
1161299451,5cyclictest1533-21ntp_kernel_pll_08:10:293
1161199457,26cyclictest7850irq/56-dwc_otg_07:35:242
11611994531,8cyclictest7850irq/56-dwc_otg_09:27:522
11611994528,4cyclictest7750irq/56-dwc_otg11:42:392
1161199450,36cyclictest7750irq/56-dwc_otg11:30:192
11611994417,16cyclictest7950irq/56-dwc_otg_10:50:272
11610994413,19cyclictest29324-21taskset07:58:441
11610994412,11cyclictest0-21swapper/108:10:071
91092430,2sleep211608-21cyclictest10:10:512
1161199435,22cyclictest24204-21/usr/sbin/munin10:30:212
1161199427,23cyclictest0-21swapper/208:30:202
1161199427,21cyclictest0-21swapper/212:05:202
11611994221,8cyclictest7750irq/56-dwc_otg12:23:152
11611994221,8cyclictest7750irq/56-dwc_otg12:23:152
11611994210,14cyclictest7950irq/56-dwc_otg_12:15:282
11610994220,11cyclictest31145-21rm08:03:551
1161199419,17cyclictest301ktimersoftd/212:15:062
1161199417,21cyclictest0-21swapper/210:05:212
1161199415,22cyclictest14540-21ssh09:39:172
11611994113,14cyclictest7850irq/56-dwc_otg_11:03:572
1161099412,9cyclictest0-21swapper/108:40:191
1161099412,9cyclictest0-21swapper/108:40:191
11610994115,12cyclictest31516-21cut08:05:191
11610994114,20cyclictest18387-21runrttasks07:27:181
11611994020,11cyclictest23647-21aten_rbpower_en09:50:212
11611994017,8cyclictest7750irq/56-dwc_otg12:39:392
11610994027,10cyclictest4488-21apt-get08:20:201
11610994015,8cyclictest16921-21sshd08:54:351
1161199395,20cyclictest24882-21ssh11:10:122
1161199393,9cyclictest0-21swapper/209:24:112
11611993927,5cyclictest18403-21ssh09:43:472
1161199392,3cyclictest0-21swapper/208:00:172
1161199392,23cyclictest21816-21df_inode11:45:232
11611993915,13cyclictest301ktimersoftd/210:55:312
11610993910,18cyclictest129752chrt07:12:301
11611993819,11cyclictest7950irq/56-dwc_otg_07:35:062
11611993814,9cyclictest7850irq/56-dwc_otg_08:30:052
11611993814,10cyclictest7950irq/56-dwc_otg_09:05:212
11611993812,8cyclictest301ktimersoftd/210:41:132
11611993812,7cyclictest7850irq/56-dwc_otg_09:13:452
1161199375,17cyclictest7750irq/56-dwc_otg12:25:262
1161199374,19cyclictest301ktimersoftd/207:45:292
11611993715,4cyclictest7750irq/56-dwc_otg08:35:272
11611993715,14cyclictest1645-21aten_rbpower_po12:00:222
11611993715,11cyclictest301ktimersoftd/211:56:222
1161199366,11cyclictest0-21swapper/207:25:252
1161199363,5cyclictest1235-21ssh10:01:352
1161199362,26cyclictest301ktimersoftd/207:20:072
1161199362,13cyclictest0-21swapper/207:13:362
11611993612,6cyclictest15982-21aten_rbpower_en10:20:212
1161199359,15cyclictest7950irq/56-dwc_otg_08:49:202
11611993514,6cyclictest7850irq/56-dwc_otg_10:46:512
11611993514,6cyclictest7850irq/56-dwc_otg_07:58:452
1161199351,25cyclictest0-21swapper/207:50:192
11611993510,6cyclictest301ktimersoftd/208:55:302
1161199345,6cyclictest7750irq/56-dwc_otg11:15:212
11611993414,12cyclictest301ktimersoftd/209:05:062
1161199337,9cyclictest0-21swapper/207:24:512
1161199337,9cyclictest0-21swapper/207:24:512
1161199336,14cyclictest0-21swapper/208:05:192
1161199336,14cyclictest0-21swapper/208:05:192
1161199333,17cyclictest301ktimersoftd/208:41:152
1161199333,17cyclictest301ktimersoftd/208:41:152
11611993312,5cyclictest6044-21modprobe08:25:052
108362338,14sleep10-21swapper/107:05:361
1161199321,7cyclictest0-21swapper/208:10:232
11611993111,6cyclictest301ktimersoftd/208:16:432
1161199301,13cyclictest0-21swapper/208:54:352
98032287,11sleep20-21swapper/207:05:212
109752284,14sleep30-21swapper/307:05:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional