You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 21:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri May 22, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10175242310,394sleep010243-21latency_hist08:08:590
10175242310,394sleep010243-21latency_hist08:08:590
30012238512,358sleep330056-21ntpq10:54:223
8534237910,335sleep08634-21wc09:44:380
286542354156,173sleep00-21swapper/012:34:180
120942332143,171sleep012213-21timerwakeupswit11:39:290
28603231628,250sleep028697-21grep10:49:260
23139229911,274sleep223171-21unin-run10:33:582
23139229911,274sleep223171-21unin-run10:33:582
18658226419,226sleep018720-21processes08:34:270
325632261110,21sleep00-21swapper/007:34:300
15088225534,203sleep015141-21mailstats11:49:270
20317219111,162sleep020390-21switchtime08:39:280
20317219111,162sleep020390-21switchtime08:39:280
25474218326,83sleep00-21swapper/012:24:120
31150218014,22sleep00-21swapper/007:29:290
2002821809,19sleep00-21swapper/010:23:580
2002821809,19sleep00-21swapper/010:23:580
13312217310,18sleep00-21swapper/008:19:010
2869021699,20sleep00-21swapper/007:24:000
2869021699,20sleep00-21swapper/007:24:000
22568216532,59sleep022604-21awk08:48:570
17058216513,69sleep00-21swapper/010:14:010
17058216513,69sleep00-21swapper/010:14:010
24181216328,120sleep224193-21latency_hist08:54:002
926216210,66sleep00-21swapper/011:04:280
31172216095,18sleep00-21swapper/009:14:250
31172216095,18sleep00-21swapper/009:14:250
2482215912,132sleep02496-21awk09:24:390
2482215912,132sleep02496-21awk09:24:390
17684215311,99sleep00-21swapper/011:59:120
6468215116,117sleep36526-21idleruntime09:39:203
418421519,101sleep00-21swapper/007:49:120
418421519,101sleep00-21swapper/007:49:120
10128215010,121sleep09380-21/usr/sbin/munin09:49:380
12504214911,97sleep012514-21find09:59:070
12504214911,97sleep012514-21find09:59:070
10502214916,40sleep00-21swapper/011:34:290
23899214820,88sleep00-21swapper/012:19:070
4949214725,102sleep34728-21apt-get11:19:013
1989214712,97sleep01840-21/usr/sbin/munin11:09:170
263821448,49sleep00-21swapper/007:44:150
2495221378,95sleep0363-21snmpd07:12:490
1173721377,42sleep00-21swapper/009:54:390
10258213613,107sleep210303-21ntpq11:34:222
5545213032,85sleep05566-21sed11:19:470
3583212924,64sleep00-21swapper/009:29:310
1355021278,79sleep00-21swapper/008:19:130
1355021278,79sleep00-21swapper/008:19:130
27007212419,87sleep327043-21sendmail07:18:583
27007212419,87sleep327043-21sendmail07:18:583
17758212412,85sleep00-21swapper/010:14:420
17140212434,75sleep00-21swapper/008:29:470
5263212314,91sleep15231-21apt-get11:19:271
30177212217,27sleep029617-21/usr/sbin/munin10:54:270
23604212011,92sleep223674-21true12:18:572
3253621197,22sleep00-21swapper/011:04:000
2331021199,99sleep00-21swapper/012:15:200
2478821157,24sleep024939-21date10:39:000
2478821157,24sleep024939-21date10:39:000
28153211410,42sleep00-21swapper/009:04:280
1408721148,16sleep00-21swapper/010:04:120
1408721148,16sleep00-21swapper/010:04:120
23263211337,46sleep123346-21unin-run07:08:581
17361211315,29sleep00-21swapper/011:58:560
1301621138,87sleep00-21swapper/011:44:140
860421128,44sleep08624-21munin-run08:03:580
29729211221,69sleep129759-21fschecks_count10:54:161
28130211212,15sleep00-21swapper/012:32:040
21270211210,89sleep121271-21sshd10:25:041
21270211210,89sleep121271-21sshd10:25:041
6304211112,59sleep00-21swapper/009:39:120
724621109,61sleep00-21swapper/011:24:230
2186821097,15sleep00-21swapper/010:29:110
2186821097,15sleep00-21swapper/010:29:110
2599321089,40sleep00-21swapper/008:59:110
24516210818,75sleep10-21swapper/108:54:191
23218210843,40sleep023248-21munin-run07:08:570
17362210716,73sleep117368-21modprobe11:58:571
1554121077,36sleep00-21swapper/008:24:260
2125421058,85sleep021283-21ps12:09:260
1651210510,78sleep01727-21sed07:39:270
2783521048,34sleep00-21swapper/007:19:280
2783521048,34sleep00-21swapper/007:19:280
23172210415,75sleep323240-21latency_hist10:33:593
23172210415,75sleep323240-21latency_hist10:33:593
28006210326,52sleep228144-21grep07:19:302
28006210326,52sleep228144-21grep07:19:302
572421028,38sleep00-21swapper/007:54:090
2527521026,35sleep00-21swapper/010:39:210
2527521026,35sleep00-21swapper/010:39:210
14563210115,72sleep214567-21grep11:49:132
14339210113,72sleep214478-21latency_hist11:49:002
30365210022,41sleep10-21swapper/109:13:581
26112210023,59sleep00-21swapper/007:14:230
26112210023,59sleep00-21swapper/007:14:230
4862998,12sleep10-21swapper/109:19:271
428029924,51sleep14389-21fschecks_time07:49:171
428029924,51sleep14389-21fschecks_time07:49:171
278062999,30sleep00-21swapper/010:48:580
278062999,30sleep00-21swapper/010:48:580
2441129915,60sleep224480-21mailstats12:19:262
204629917,67sleep32085-21wc09:24:343
204629917,67sleep32085-21wc09:24:343
1744229910,77sleep317506-21latency_hist11:58:593
104132997,34sleep00-21swapper/008:09:110
253572985,52sleep00-21swapper/008:54:510
2157629816,62sleep321592-21/usr/sbin/munin08:44:203
1893429712,66sleep218988-21fschecks_time10:19:322
1893429712,66sleep218988-21fschecks_time10:19:322
1693829719,59sleep217006-21latency08:29:412
2779229517,22sleep30-21swapper/307:19:273
2779229517,22sleep30-21swapper/307:19:273
277582957,13sleep30-21swapper/309:04:193
162729516,62sleep31775-21latency_hist11:08:593
193512946,46sleep00-21swapper/012:04:170
1061929416,57sleep110762-21sed11:34:301
1768729310,71sleep317754-21readlink10:14:413
1447429313,18sleep30-21swapper/310:04:313
1447429313,18sleep30-21swapper/310:04:313
1983629211,22sleep319844-21sh08:39:153
1983629211,22sleep319844-21sh08:39:153
3140829119,20sleep30-21swapper/309:14:333
3140829119,20sleep30-21swapper/309:14:333
46212906,28sleep00-21swapper/011:18:090
2203929027,43sleep322067-21grep12:13:573
1153629015,53sleep20-21swapper/209:54:372
1012328915,59sleep210138-21idleruntime-cro08:08:572
1012328915,59sleep210138-21idleruntime-cro08:08:572
57632885,29sleep00-21swapper/009:36:100
3159028816,18sleep30-21swapper/310:59:223
212028713,14sleep10-21swapper/107:41:051
1932428727,28sleep30-21swapper/312:04:163
842828620,53sleep18463-21fschecks_count11:29:171
241282868,15sleep00-21swapper/008:53:580
1524128614,56sleep215303-21fw_packets08:24:202
1109628624,38sleep311234-21unixbench_multi08:09:313
3017328518,53sleep230206-21date07:28:592
3017328518,53sleep230206-21date07:28:592
2363228315,46sleep323684-21/usr/sbin/munin10:34:193
2363228315,46sleep323684-21/usr/sbin/munin10:34:193
211162837,34sleep00-21swapper/010:24:310
211162837,34sleep00-21swapper/010:24:310
1777828319,40sleep117868-21grep11:59:171
1577628321,26sleep10-21swapper/108:24:311
1944628219,42sleep219447-21cron08:38:582
2253328130,26sleep20-21swapper/207:04:282
2910128012,48sleep10-21swapper/107:24:171
2910128012,48sleep10-21swapper/107:24:171
1279528019,36sleep212939-21ls11:44:002
95927918,28sleep00-21swapper/009:19:460
2737127910,15sleep10-21swapper/109:04:001
83412788,32sleep00-21swapper/011:29:120
2352227818,34sleep30-21swapper/307:09:013
78727717,27sleep20-21swapper/211:04:262
3025727718,28sleep00-21swapper/009:12:360
284327724,27sleep10-21swapper/107:44:201
185827713,36sleep20-21swapper/209:24:292
185827713,36sleep20-21swapper/209:24:292
604827510,28sleep20-21swapper/211:19:572
1634927527,28sleep30-21swapper/308:28:573
793327412,46sleep27940-21df_inode09:44:232
289112747,14sleep30-21swapper/310:49:293
2887127416,30sleep30-21swapper/307:24:113
2887127416,30sleep30-21swapper/307:24:113
272127415,37sleep32738-21fschecks_count07:44:183
256962749,15sleep20-21swapper/212:24:182
2419427319,28sleep30-21swapper/308:54:013
2270927314,42sleep322752-21ntpq12:14:223
2098727329,22sleep30-21swapper/310:24:303
2098727329,22sleep30-21swapper/310:24:303
2050727317,37sleep220657-21grep08:39:302
2050727317,37sleep220657-21grep08:39:302
1675427314,25sleep30-21swapper/308:29:383
1276227322,28sleep10-21swapper/111:43:591
488027226,27sleep20-21swapper/209:34:262
3266227222,26sleep20-21swapper/209:19:202
296862724,56sleep20-21swapper/212:37:102
2429527215,33sleep20-21swapper/207:09:322
726627112,25sleep20-21swapper/211:24:252
2552627117,39sleep3225-21systemd-journal08:57:043
1379527115,35sleep2284-21dbus-daemon10:03:582
1379527115,35sleep2284-21dbus-daemon10:03:582
2940327022,29sleep129410-21cat10:53:591
940526915,29sleep30-21swapper/309:49:133
781726914,40sleep37818-21sshd09:44:043
3131526923,23sleep20-21swapper/210:59:172
311752699,17sleep10-21swapper/107:29:301
1394626926,26sleep10-21swapper/110:04:011
1394626926,26sleep10-21swapper/110:04:011
126926921,28sleep30-21swapper/307:39:183
3226226813,31sleep132337-21/usr/sbin/munin07:34:231
3136926822,25sleep20-21swapper/209:14:322
3136926822,25sleep20-21swapper/209:14:322
2785626819,24sleep10-21swapper/110:48:591
2785626819,24sleep10-21swapper/110:48:591
2678126819,19sleep30-21swapper/312:29:003
2652226815,35sleep326541-21apt-get10:44:083
2652226815,35sleep326541-21apt-get10:44:083
2509326816,24sleep10-21swapper/107:13:591
15582687,17sleep10-21swapper/111:08:451
1135926817,23sleep10-21swapper/109:54:321
1041426818,31sleep110409-21apt-get08:09:121
306426720,23sleep20-21swapper/207:44:262
2007426719,25sleep10-21swapper/110:23:591
2007426719,25sleep10-21swapper/110:23:591
1470926714,22sleep10-21swapper/110:04:371
1470926714,22sleep10-21swapper/110:04:371
270392666,13sleep30-21swapper/309:01:083
85026522,19sleep20-21swapper/207:39:002
304232659,33sleep20-21swapper/209:13:592
2976326524,21sleep20-21swapper/210:54:172
2729226521,24sleep20-21swapper/212:29:192
1236826518,21sleep20-21swapper/208:14:262
996526410,24sleep20-21swapper/209:49:362
863526420,23sleep10-21swapper/109:44:391
857326420,28sleep38597-21awk08:03:573
687626418,20sleep10-21swapper/111:24:161
573126410,26sleep30-21swapper/307:54:123
405826422,20sleep20-21swapper/211:14:282
2574426423,20sleep10-21swapper/110:39:301
2574426423,20sleep10-21swapper/110:39:301
2174126415,25sleep20-21swapper/210:29:012
2174126415,25sleep20-21swapper/210:29:012
880426315,20sleep20-21swapper/211:29:262
489626324,21sleep30-21swapper/309:34:273
385926317,27sleep23979-21sed09:29:382
277732636,13sleep20-21swapper/210:48:352
277732636,13sleep20-21swapper/210:48:352
2589226317,29sleep225906-21/usr/sbin/munin10:39:312
2589226317,29sleep225906-21/usr/sbin/munin10:39:312
1688726322,23sleep20-21swapper/211:54:302
1319726314,34sleep113255-21if_err_eth011:44:191
989826220,25sleep10-21swapper/109:49:351
3130426221,18sleep10-21swapper/110:59:161
21512629,26sleep30-21swapper/311:09:203
1958626217,23sleep10-21swapper/108:39:011
1958626217,23sleep10-21swapper/108:39:011
1265426217,29sleep112808-21unixbench_multi08:14:331
2523126120,18sleep30-21swapper/312:23:593
234182615,11sleep20-21swapper/210:34:142
234182615,11sleep20-21swapper/210:34:142
2214326115,28sleep222212-21/usr/sbin/munin08:44:312
2118726121,19sleep10-21swapper/112:09:231
1725326119,21sleep10-21swapper/110:14:141
548526015,23sleep20-21swapper/207:53:592
548526015,23sleep20-21swapper/207:53:592
3131726022,22sleep20-21swapper/207:29:312
2593826020,23sleep10-21swapper/112:24:251
1630726023,20sleep30-21swapper/310:09:403
1630726023,20sleep30-21swapper/310:09:403
1503026018,28sleep30-21swapper/311:49:263
66442598,12sleep20-21swapper/207:54:312
2827425913,24sleep30-21swapper/312:33:563
2544825913,32sleep225543-21ntp_states07:14:042
2544825913,32sleep225543-21ntp_states07:14:042
183412596,17sleep20-21swapper/210:17:342
1068625916,27sleep210762-21sed08:09:212
594025816,20sleep10-21swapper/107:54:181
523325813,27sleep34728-21apt-get11:19:093
476425817,26sleep10-21swapper/111:18:591
3223525821,21sleep10-21swapper/109:18:591
3223525821,21sleep10-21swapper/109:18:591
2339425819,20sleep10-21swapper/108:49:271
2167925817,22sleep10-21swapper/108:44:221
2015925818,24sleep320222-21cat10:24:013
2015925818,24sleep320222-21cat10:24:013
1887925815,21sleep30-21swapper/308:34:303
332925720,21sleep30-21swapper/309:29:263
326425715,16sleep10-21swapper/109:29:201
739625611,18sleep20-21swapper/207:59:202
657425612,23sleep10-21swapper/109:39:221
53322565,12sleep30-21swapper/307:52:343
53322565,12sleep30-21swapper/307:52:343
3051225614,26sleep330654-21latency_hist09:14:003
2429425616,19sleep10-21swapper/112:19:201
2409525615,24sleep124213-21sed10:34:291
2409525615,24sleep124213-21sed10:34:291
1795125616,25sleep10-21swapper/108:34:001
1485525615,18sleep30-21swapper/308:24:003
1485525615,18sleep30-21swapper/308:24:003
1014225621,19sleep30-21swapper/308:08:583
1014225621,19sleep30-21swapper/308:08:583
960925516,22sleep19747-21sed08:04:321
960925516,22sleep19747-21sed08:04:321
5002556,18sleep30-21swapper/307:35:343
2755125514,25sleep227621-21df09:04:142
2597725514,25sleep126011-21irqstats07:14:201
2597725514,25sleep126011-21irqstats07:14:201
2375525513,24sleep30-21swapper/307:09:093
2013625517,19sleep10-21swapper/112:04:301
1872225515,23sleep10-21swapper/108:34:281
186712557,13sleep20-21swapper/212:00:232
1694125519,17sleep20-21swapper/210:13:592
1694125519,17sleep20-21swapper/210:13:592
213225411,23sleep20-21swapper/211:09:192
187042546,12sleep30-21swapper/312:00:493
153332547,36sleep20-21swapper/210:08:502
153332547,36sleep20-21swapper/210:08:502
1456225412,24sleep114556-21apt-get11:49:121
1161725418,18sleep30-21swapper/309:54:383
988625312,24sleep39882-21apt-get11:34:113
3176325316,14sleep30-21swapper/307:34:013
2599425312,24sleep125988-21apt-get08:59:121
1923925311,25sleep219231-21apt-get12:04:112
1328325316,23sleep30-21swapper/311:44:213
637925213,23sleep26385-21diskstats09:39:142
157525212,22sleep11569-21apt-get09:24:111
157525212,22sleep11569-21apt-get09:24:111
1281325220,18sleep30-21swapper/309:59:303
1281325220,18sleep30-21swapper/309:59:303
1201825218,15sleep30-21swapper/311:39:283
782025114,24sleep11-21systemd07:59:311
534225118,15sleep10-21swapper/109:34:371
2792825114,17sleep10-21swapper/107:19:291
2792825114,17sleep10-21swapper/107:19:291
2312125111,25sleep123134-21idleruntime-cro10:33:571
2312125111,25sleep123134-21idleruntime-cro10:33:571
2241625115,21sleep122444-21fschecks_count12:14:161
296372506,34sleep30-21swapper/312:36:353
2903025014,16sleep10-21swapper/112:34:271
2608825014,18sleep20-21swapper/208:59:172
1354125012,22sleep113544-21apt-get08:19:101
1354125012,22sleep113544-21apt-get08:19:101
686624912,22sleep36869-21df_inode11:24:143
2807024811,21sleep228089-21apt-get10:49:082
30542476,10sleep10-21swapper/111:13:051
2448624711,21sleep224497-21df_inode08:54:162
2136324714,19sleep20-21swapper/212:09:282
1204924712,20sleep312050-21diskstats08:14:193
873924617,15sleep30-21swapper/311:29:233
44124510,20sleep3477-21ntp_states09:19:253
2701924510,19sleep127014-21apt-get12:29:101
137592437,16sleep20-21swapper/208:19:192
137592437,16sleep20-21swapper/208:19:192
117924310,14sleep20-21swapper/207:39:172
250012429,15sleep30-21swapper/310:39:113
250012429,15sleep30-21swapper/310:39:113
1570024011,15sleep10-21swapper/110:09:131
1570024011,15sleep10-21swapper/110:09:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional