You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 11:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jan 01, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4656293411,908sleep324090-21kworker/u8:321:59:133
26307286811,840sleep326336-21/usr/sbin/munin21:24:173
21871265819,623sleep121822-21/usr/sbin/munin00:38:561
23505256412,532sleep023646-21unixbench_multi21:14:280
12141251521,441sleep012291-21cut22:23:570
15985247627,431sleep116053-21/usr/sbin/munin22:34:231
20641245127,404sleep320709-21/usr/sbin/munin22:49:243
18464240268,202sleep018477-21cron00:28:530
17099238623,348sleep317160-21/usr/sbin/munin22:39:143
916723769,41sleep00-21swapper/020:29:190
28989237640,324sleep029033-21ntpq19:49:180
32759233469,107sleep00-21swapper/023:29:140
21035229215,221sleep021081-21latency_hist21:08:550
21035229215,221sleep021081-21latency_hist21:08:550
20760227516,184sleep00-21swapper/000:34:510
214922716,11sleep00-21swapper/020:06:580
4874226322,218sleep25023-21unixbench_singl20:14:272
30981223917,205sleep231049-21/usr/sbin/munin19:55:092
6803223512,206sleep26827-21df_inode23:49:142
17728223356,69sleep017874-21unixbench_singl22:39:270
4049222714,196sleep11-21systemd21:58:541
18208222018,188sleep018236-21fschecks_count20:59:140
18208222018,188sleep018236-21fschecks_count20:59:140
28576221934,169sleep028712-21grep23:14:280
20007221229,136sleep020055-21systemd-run00:33:540
20007221229,136sleep020055-21systemd-run00:33:540
4002221138,59sleep00-21swapper/023:40:160
4002221138,59sleep00-21swapper/023:40:160
4002221138,59sleep00-21swapper/023:40:160
2675221147,105sleep00-21swapper/020:09:160
2675221147,105sleep00-21swapper/020:09:160
7052220752,135sleep0369-21snmpd20:23:580
30330220526,143sleep029857-21apt-get19:53:580
18094220569,124sleep018108-21needreboot19:14:290
14351220212,175sleep014437-21cut00:14:240
14351220212,175sleep014437-21cut00:14:240
21507219758,119sleep00-21swapper/019:24:270
7772219610,124sleep07865-21fschecks_time22:09:150
14508219654,125sleep014639-21timerwakeupswit22:29:260
4030218818,115sleep04043-21modprobe21:58:530
8653218658,71sleep00-21swapper/023:54:170
24510217458,51sleep00-21swapper/019:34:280
13531217410,41sleep00-21swapper/020:44:130
31070217040,114sleep031108-21processes21:39:230
2485221615,132sleep00-21swapper/023:04:180
1552216027,115sleep31571-21basename23:33:533
11671215918,10sleep00-21swapper/000:07:420
16917215813,118sleep017110-21sed19:13:570
29398215621,75sleep00-21swapper/021:34:190
22582215636,70sleep022681-21latency_hist21:13:560
25794215211,123sleep025830-21latency19:39:330
16636215217,115sleep016693-21fschecks_count20:54:140
6512215028,107sleep06557-21ntpq22:04:180
26930215017,113sleep027016-21sort23:09:430
1078721469,121sleep00-21swapper/020:34:240
24473214336,90sleep00-21swapper/023:03:540
5323214017,103sleep00-21swapper/023:44:160
2388214023,96sleep00-21swapper/023:34:230
9957213912,21sleep30-21swapper/323:59:173
24026213724,86sleep124084-21fschecks_time19:34:171
17954213623,89sleep118092-21head00:24:431
3240821327,85sleep00-21swapper/019:59:230
14848213120,31sleep30-21swapper/319:04:273
14848213120,31sleep30-21swapper/319:04:273
20182212957,20sleep00-21swapper/022:49:120
99621288,97sleep0994-21munin-run21:48:560
2458521288,107sleep00-21swapper/021:19:170
11180212819,92sleep211293-21cat00:03:562
7359212430,48sleep00-21swapper/023:49:290
12431212414,86sleep312498-21fschecks_time22:24:123
29107212316,92sleep229142-21sendmail23:18:542
10516212346,23sleep00-21swapper/023:59:300
6375212212,90sleep16113-21/usr/sbin/munin22:04:151
22934212112,94sleep123003-21awk19:29:241
831321205,10sleep00-21swapper/020:26:200
1608211941,64sleep01649-21ntpq21:49:160
1608211941,64sleep01649-21ntpq21:49:160
3634211813,91sleep33127-21apt-get23:39:063
3634211813,91sleep33127-21apt-get23:39:063
3634211813,91sleep33127-21apt-get23:39:063
1961121176,12sleep00-21swapper/022:46:180
14131211611,89sleep113428-21/usr/sbin/munin20:44:261
31279211512,31sleep231285-21df_inode23:24:112
20507211428,64sleep320022-21apt-get00:34:053
19392211323,70sleep119431-21fschecks_count19:19:151
15509210928,56sleep215529-21apt-get19:09:062
3097210828,58sleep33127-21apt-get23:38:543
13923210816,73sleep213986-21fschecks_time00:14:142
13923210816,73sleep213986-21fschecks_time00:14:142
17004210632,47sleep116693-21apt-get00:23:581
30707210421,67sleep330766-21latency19:55:023
2033021035,56sleep00-21swapper/019:20:550
6223210216,63sleep26230-21/usr/sbin/munin22:04:122
519621025,60sleep00-21swapper/020:16:190
12282210017,68sleep211794-21apt-get00:09:052
3040429912,71sleep230555-21date21:38:572
1810929914,67sleep118151-21sed19:14:301
1102229912,67sleep111148-21/usr/sbin/munin20:34:291
729429718,64sleep36723-21/usr/sbin/munin23:49:283
2467229514,69sleep224739-21latency21:19:192
271412936,75sleep00-21swapper/021:28:090
233882937,74sleep0369-21snmpd19:30:320
2757429119,60sleep00-21swapper/021:29:140
2757429119,60sleep00-21swapper/021:29:140
440329019,20sleep10-21swapper/120:14:181
3223129016,53sleep232303-21fschecks_time21:44:142
152529015,15sleep30-21swapper/320:04:233
160382897,71sleep0369-21snmpd20:50:590
1229528923,18sleep00-21swapper/000:09:110
333528819,51sleep23450-21sed23:38:572
325228821,47sleep23314-21grep21:54:232
229528826,44sleep2228-21systemd-journal20:08:542
2169728818,54sleep321718-21df_inode22:54:123
2169728818,54sleep321718-21df_inode22:54:123
1235028812,54sleep112258-21/usr/sbin/munin00:09:531
1231428830,19sleep012361-21ntpq20:39:200
296562879,61sleep20-21swapper/223:19:062
293022879,19sleep30-21swapper/321:34:183
2769628714,55sleep227746-21if_eth021:29:162
2769628714,55sleep227746-21if_eth021:29:162
2108328615,16sleep10-21swapper/121:08:561
166002865,11sleep00-21swapper/000:22:520
3164128520,14sleep00-21swapper/023:24:210
2772028521,16sleep00-21swapper/019:44:250
2772028521,16sleep00-21swapper/019:44:250
2704128524,40sleep227038-21apt-get19:44:072
2704128524,40sleep227038-21apt-get19:44:072
178728522,20sleep10-21swapper/121:49:211
178728522,20sleep10-21swapper/121:49:211
118582850,6sleep20-21swapper/220:38:572
2943028413,57sleep129580-21cat23:18:571
2512828414,51sleep124743-21/usr/sbin/munin23:04:241
2189028331,33sleep00-21swapper/022:54:170
2189028331,33sleep00-21swapper/022:54:170
1475928322,37sleep314737-21latency_hist20:48:563
2094428217,44sleep121049-21fschecks_time19:24:151
1416028218,46sleep014217-21interrupts19:04:160
1416028218,46sleep014217-21interrupts19:04:160
649628113,21sleep30-21swapper/323:48:553
3068928115,46sleep330731-21fschecks_count21:39:143
2266028119,46sleep222733-21latency19:29:172
568928014,50sleep25762-21tr23:44:262
2729028024,23sleep30-21swapper/321:28:573
2729028024,23sleep30-21swapper/321:28:573
2589328019,46sleep225924-21nfs_client19:39:352
1892128020,46sleep318933-21netstat22:44:193
1831828025,23sleep10-21swapper/122:43:561
1056928012,53sleep10-21swapper/122:18:571
2970327929,35sleep00-21swapper/023:19:370
224142796,13sleep20-21swapper/221:12:242
1210127914,50sleep312136-21cat22:23:563
1077927916,48sleep110780-21/usr/sbin/munin22:19:051
481327820,12sleep00-21swapper/021:59:260
2209827827,34sleep322126-21munin-run19:28:543
901927713,49sleep19172-21latency_hist22:13:561
2449227623,27sleep30-21swapper/321:19:163
2012027615,45sleep120153-21/usr/sbin/munin21:04:201
1508627622,25sleep10-21swapper/120:49:141
1260427618,14sleep20-21swapper/220:39:262
920027516,23sleep20-21swapper/222:14:052
965827418,28sleep30-21swapper/323:58:563
245527418,40sleep12570-21sed23:34:241
2085527418,34sleep20-21swapper/200:34:532
1998127410,49sleep219995-21irqstats21:04:172
1998127410,49sleep219995-21irqstats21:04:172
640627318,24sleep10-21swapper/120:19:271
164627320,20sleep20-21swapper/223:33:562
1015927326,11sleep00-21swapper/022:14:440
295727224,28sleep10-21swapper/120:09:211
295727224,28sleep10-21swapper/120:09:211
2495027223,30sleep20-21swapper/223:04:202
1499727210,46sleep30-21swapper/320:49:083
2561127125,22sleep10-21swapper/119:39:301
2452327121,25sleep20-21swapper/223:03:562
177112717,16sleep20-21swapper/219:14:072
1330327126,11sleep00-21swapper/022:25:020
424727023,26sleep30-21swapper/321:58:563
2653827023,24sleep10-21swapper/121:24:231
166962709,18sleep00-21swapper/022:38:550
1063927019,23sleep10-21swapper/123:59:311
52042694,14sleep30-21swapper/320:16:263
2859026916,25sleep30-21swapper/319:49:093
22762696,11sleep20-21swapper/221:51:082
22762696,11sleep20-21swapper/221:51:082
1833026918,26sleep20-21swapper/220:59:162
1833026918,26sleep20-21swapper/220:59:162
1544126924,24sleep30-21swapper/322:34:093
739126819,35sleep37418-21munin-plugin-st22:08:543
3130626814,31sleep10-21swapper/123:24:141
2594026816,23sleep20-21swapper/221:24:072
1883926826,27sleep30-21swapper/300:29:133
1883926826,27sleep30-21swapper/300:29:133
3251326722,18sleep30-21swapper/321:44:193
3214326715,31sleep132137-21apt-get21:44:091
2210326717,29sleep20-21swapper/222:54:232
2210326717,29sleep20-21swapper/222:54:232
1882726721,24sleep20-21swapper/222:44:182
1441126721,25sleep20-21swapper/219:04:222
1441126721,25sleep20-21swapper/219:04:222
1440326717,34sleep113926-21/usr/sbin/munin19:04:211
1440326717,34sleep113926-21/usr/sbin/munin19:04:211
645326618,23sleep10-21swapper/123:48:531
2685726621,23sleep30-21swapper/323:09:423
2518426615,28sleep30-21swapper/319:38:553
2236326621,27sleep322382-21apt-get19:29:063
1901226614,31sleep20-21swapper/200:29:162
1901226614,31sleep20-21swapper/200:29:162
1005026623,27sleep30-21swapper/320:33:543
3195026512,37sleep132018-21df19:59:121
1781526512,20sleep30-21swapper/319:14:243
1598926514,35sleep115995-21ntp_states19:09:211
1295626511,36sleep213072-21sed22:24:232
453626413,33sleep24633-21users23:40:262
453626413,33sleep24633-21users23:40:262
453626413,33sleep24633-21users23:40:262
2926926417,30sleep229422-21awk19:49:242
2917426422,24sleep329206-21munin-run23:18:553
2704526419,20sleep10-21swapper/119:44:101
2704526419,20sleep10-21swapper/119:44:101
2319126413,34sleep123222-21/usr/sbin/munin21:14:211
1695626417,20sleep20-21swapper/220:54:202
1157226417,18sleep20-21swapper/222:19:272
855526319,18sleep20-21swapper/223:54:152
3190726319,22sleep10-21swapper/121:43:561
1437426320,19sleep20-21swapper/222:29:242
1237826311,29sleep30-21swapper/320:39:233
515926227,18sleep10-21swapper/121:59:351
2484126218,28sleep324850-21/usr/sbin/munin23:04:173
1602126223,22sleep30-21swapper/319:09:233
1182626218,26sleep111827-21/usr/sbin/munin00:08:561
1084526221,27sleep30-21swapper/320:34:273
675226121,26sleep16802-21/usr/sbin/munin23:49:131
2091626118,25sleep220923-21df_inode19:24:122
1910226111,32sleep119170-21/usr/sbin/munin22:44:251
1273426123,23sleep30-21swapper/300:10:153
856326021,15sleep30-21swapper/320:28:573
303642609,17sleep10-21swapper/119:54:071
2841426020,22sleep128554-21modprobe19:48:581
882225911,34sleep38823-21sshd22:12:253
725625915,27sleep17249-21apt-get20:24:111
611725914,20sleep30-21swapper/320:19:213
3255425920,18sleep30-21swapper/319:59:263
2878125918,26sleep128791-21unin-run21:33:551
2878125918,26sleep128791-21unin-run21:33:551
2379325916,23sleep30-21swapper/322:59:293
1930725912,32sleep219375-21df19:19:122
1728325926,19sleep20-21swapper/222:39:172
1679225918,18sleep20-21swapper/222:38:572
1672125918,24sleep216817-21apt-key00:23:552
131725915,26sleep31412-21perl21:49:113
131725915,26sleep31412-21perl21:49:113
590125823,19sleep20-21swapper/222:03:562
2493625813,27sleep125009-21sensors_temp21:19:261
954025716,25sleep1287-21dbus-daemon23:58:541
906725711,29sleep29135-21ls20:29:172
369425715,16sleep10-21swapper/123:39:361
369425715,16sleep10-21swapper/123:39:361
369425715,16sleep10-21swapper/123:39:361
298325714,20sleep20-21swapper/220:09:222
298325714,20sleep20-21swapper/220:09:222
194052576,11sleep10-21swapper/121:03:311
194052576,11sleep10-21swapper/121:03:311
1383625711,26sleep113830-21apt-get00:14:091
1383625711,26sleep113830-21apt-get00:14:091
3096825617,19sleep30-21swapper/323:23:563
2154525616,25sleep10-21swapper/121:09:171
2078125622,17sleep10-21swapper/122:49:261
1720825614,25sleep217205-21apt-get00:24:072
1709325615,19sleep30-21swapper/320:54:253
1398825622,20sleep30-21swapper/300:14:153
1398825622,20sleep30-21swapper/300:14:153
3036525519,17sleep10-21swapper/121:38:561
2805225514,20sleep10-21swapper/123:14:171
1190825515,15sleep20-21swapper/200:08:572
234092548,31sleep123464-21if_eth022:59:191
1499625414,24sleep214993-21apt-get20:49:072
3251625317,19sleep332491-21latency_hist23:28:563
316425311,26sleep33234-21switchtime20:09:273
316425311,26sleep33234-21switchtime20:09:273
2408725312,23sleep20-21swapper/219:34:182
2039925314,25sleep319700-21/usr/sbin/munin21:04:263
2039925314,25sleep319700-21/usr/sbin/munin21:04:263
1768625311,27sleep317692-21ntp_states00:24:363
1426325312,23sleep214354-21users20:44:272
790425213,22sleep27953-21irqstats22:09:172
44925219,18sleep10-21swapper/123:29:191
2415425216,17sleep30-21swapper/321:18:553
2231825216,21sleep122442-21sed22:54:261
2231825216,21sleep122442-21sed22:54:261
1983825210,18sleep30-21swapper/319:19:253
1538225215,17sleep30-21swapper/300:18:593
1856525113,19sleep30-21swapper/320:59:213
1856525113,19sleep30-21swapper/320:59:213
1729225115,17sleep10-21swapper/120:54:271
1411525112,21sleep314159-21irqstats22:29:173
1199225113,23sleep111997-21diskstats20:39:131
112425114,21sleep11128-21diskstats20:04:131
1055525115,16sleep30-21swapper/322:18:553
902125015,19sleep19066-21irqstats20:29:161
2631225012,21sleep226310-21apt-get23:09:212
2127125013,22sleep321269-21apt-get21:09:063
150202506,16sleep10-21swapper/122:32:171
80324915,18sleep20-21swapper/220:03:572
57132498,17sleep20-21swapper/220:19:112
72512489,15sleep20-21swapper/220:24:082
2573524813,18sleep30-21swapper/319:39:323
133742484,35sleep10-21swapper/122:25:561
2640124712,20sleep126403-21diskstats23:09:281
1875224713,20sleep118749-21apt-get00:29:071
1875224713,20sleep118749-21apt-get00:29:071
2795924614,15sleep30-21swapper/319:44:283
2795924614,15sleep30-21swapper/319:44:283
2009524613,17sleep20-21swapper/222:49:062
306842446,15sleep20-21swapper/221:39:132
238312435,28sleep20-21swapper/221:15:442
1150623510,12sleep30-21swapper/300:05:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional