You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 20:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Apr 04, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
280832822228,8159sleep323989-21kworker/3:008:57:243
194612485220,4798sleep315838-21kworker/3:208:31:113
6785211269,1107sleep022877-1kworker/0:0H11:17:110
20769264512,618sleep020762-21apt-get12:02:110
29804256715,458sleep329826-21cron12:31:553
239502497117,244sleep30-21swapper/312:12:163
28072241521,377sleep128089-21/usr/sbin/munin10:42:061
14543232217,288sleep214537-21apt-get11:42:092
26024226831,129sleep326036-21apt-get08:52:073
16226223147,162sleep316366-21latency_hist08:21:583
10449222339,122sleep30-21swapper/311:27:243
30277221411,190sleep330314-21idleruntime12:32:153
30277221411,190sleep330314-21idleruntime12:32:153
20266221116,60sleep30-21swapper/310:17:163
26226221036,93sleep30-21swapper/312:17:293
9543220676,70sleep30-21swapper/309:42:193
24053220314,174sleep124145-21cat10:28:091
24053220314,174sleep124145-21cat10:28:091
4724219047,120sleep30-21swapper/311:11:573
7388217815,143sleep37434-21latency_hist09:36:583
3446217557,75sleep30-21swapper/307:41:553
3446217557,75sleep30-21swapper/307:41:553
31352217318,139sleep231387-21idleruntime-cro12:36:552
31352217318,139sleep231387-21idleruntime-cro12:36:552
29749217313,42sleep30-21swapper/309:02:253
29749217313,42sleep30-21swapper/309:02:253
2202921577,138sleep322076-21ntpq10:22:213
12117215722,14sleep30-21swapper/308:07:203
15802215633,106sleep30-21swapper/311:46:553
14308215630,110sleep314441-21latency_hist11:41:583
22935215527,34sleep30-21swapper/308:42:103
803121538,131sleep38075-21ntp_offset09:37:223
18320215029,105sleep318460-21modprobe10:11:593
30433214813,114sleep13120-1kworker/1:2H09:06:551
30433214813,114sleep13120-1kworker/1:2H09:06:551
24971214729,80sleep325034-21sshd10:32:113
24971214729,80sleep325034-21sshd10:32:113
22372145106,18sleep30-21swapper/311:02:203
12761214346,52sleep30-21swapper/311:36:583
1967321428,119sleep019679-21ntpq11:57:220
312213915,110sleep2294-21rs:main2
523021378,94sleep35259-21cut11:12:183
523021378,94sleep35259-21cut11:12:183
32069213413,76sleep30-21swapper/309:11:583
31894213427,69sleep331930-21ntpq07:27:213
21222213438,79sleep30-21swapper/312:02:223
8040213313,83sleep38044-21modprobe11:21:553
3048213118,95sleep03133-21fschecks_time09:22:170
27859213113,104sleep027957-21latency_hist10:41:570
13385213119,89sleep213450-21sort08:12:162
29730212911,98sleep029733-21/usr/sbin/munin10:47:140
8678212713,29sleep18700-21kernelversion07:57:201
266721278,49sleep32670-21cron09:21:553
22780212735,55sleep30-21swapper/312:07:233
27034212230,51sleep327098-21awk10:37:243
18009212034,45sleep30-21swapper/311:52:203
883211884,23sleep3897-21grep07:32:223
30345211617,17sleep30-21swapper/307:22:203
23148211514,80sleep223206-21/usr/sbin/munin08:42:172
10201211538,35sleep30-21swapper/308:02:113
6968211222,20sleep30-21swapper/307:52:173
29704211114,82sleep029740-21munin-run07:21:570
23385211113,82sleep222917-21apt-get10:27:052
23385211113,82sleep222917-21apt-get10:27:052
30954210916,45sleep30-21swapper/310:51:583
9842210729,37sleep30-21swapper/308:01:593
2477221046,79sleep00-21swapper/008:47:200
3416210325,55sleep13468-21cpuspeed11:07:111
13533210220,60sleep213552-21idleruntime-cro09:56:562
27381210115,72sleep127454-21latency_hist08:56:581
2496229814,67sleep124994-21/usr/sbin/munin08:47:251
151642969,54sleep315237-21latency_hist10:01:583
2843229512,41sleep30-21swapper/307:17:113
238442957,38sleep30-21swapper/310:28:033
238442957,38sleep30-21swapper/310:28:033
1749129518,39sleep30-21swapper/310:07:233
2830929420,32sleep30-21swapper/310:42:173
254632949,72sleep325527-21switchtime07:07:303
1790329413,58sleep117614-21/usr/sbin/munin11:52:181
973129313,64sleep19792-21switchtime09:42:251
2926829318,54sleep029410-21sed12:27:280
2693629317,52sleep10-21swapper/107:12:171
1921129218,33sleep30-21swapper/311:57:093
1540129222,32sleep30-21swapper/310:02:103
492329120,34sleep30-21swapper/309:27:213
323229131,44sleep30-21swapper/309:22:203
2692329122,32sleep30-21swapper/307:12:153
119852919,46sleep312023-21modprobe09:51:563
1917629018,54sleep20-21swapper/210:12:272
3017928915,59sleep130182-21/usr/sbin/munin12:32:131
3017928915,59sleep130182-21/usr/sbin/munin12:32:131
2507528931,18sleep30-21swapper/308:47:273
382528812,37sleep30-21swapper/307:42:163
1765828811,19sleep10-21swapper/110:07:251
1502628820,47sleep014518-21/usr/sbin/munin11:42:240
52912879,41sleep35339-21awk07:47:133
1721728711,61sleep017298-21latency10:07:160
132932877,34sleep30-21swapper/308:12:093
860328624,38sleep30-21swapper/311:22:193
3822867,33sleep30-21swapper/310:57:153
1740328613,52sleep117529-21latency_hist11:51:581
1740328613,52sleep117529-21latency_hist11:51:581
2288928513,59sleep222921-21sensors12:07:262
103012845,11sleep20-21swapper/209:45:132
3126928316,29sleep30-21swapper/310:52:143
3126928316,29sleep30-21swapper/310:52:143
2678728314,54sleep026926-21latency_hist12:21:580
1873328311,25sleep00-21swapper/008:27:370
117628314,19sleep30-21swapper/309:16:593
58028216,51sleep0652-21ls10:57:200
3043728216,53sleep230449-21open_files07:22:242
2830928211,54sleep228421-21date12:26:572
973828124,26sleep20-21swapper/208:01:582
2187428123,37sleep30-21swapper/308:37:253
205112818,47sleep00-21swapper/010:17:210
1731628113,53sleep217362-21/usr/sbin/munin10:07:182
1689228118,28sleep00-21swapper/008:22:240
1689228118,28sleep00-21swapper/008:22:240
347028016,48sleep23492-21df_inode11:07:122
1162528011,55sleep211679-21if_eth011:32:162
2397927920,43sleep024052-21/usr/sbin/munin10:28:080
2397927920,43sleep024052-21/usr/sbin/munin10:28:080
1889727921,34sleep00-21swapper/010:12:210
3152427811,49sleep00-21swapper/007:27:100
2836827812,53sleep028405-21latency10:42:180
212322787,50sleep10-21swapper/112:02:241
1825427816,46sleep218360-21ls11:52:282
156632785,53sleep00-21swapper/010:02:190
3119727715,48sleep131232-21cpu10:52:121
3119727715,48sleep131232-21cpu10:52:121
262192779,54sleep226271-21idleruntime08:52:182
2527827719,27sleep20-21swapper/207:07:252
170372766,12sleep30-21swapper/311:47:563
170372766,12sleep30-21swapper/311:47:563
1355327646,18sleep3281-21dbus-daemon09:56:573
2665827512,40sleep026760-21/usr/sbin/munin08:52:290
2553127513,47sleep025654-21timerwakeupswit07:07:310
2302527518,40sleep023070-21fschecks_count08:42:150
2185227516,34sleep20-21swapper/208:37:242
1586627513,44sleep215899-21proc_pri10:02:252
1430227516,43sleep014309-21/usr/sbin/munin09:57:220
66822746,15sleep20-21swapper/211:17:012
2580427415,44sleep225938-21latency_hist08:51:582
165127416,37sleep21787-21latency_hist11:01:592
820827314,34sleep00-21swapper/009:37:270
576627322,27sleep20-21swapper/209:31:562
2521327310,48sleep025252-21sed12:16:570
2456927329,23sleep20-21swapper/207:06:572
2456927329,23sleep20-21swapper/207:06:572
1642727321,16sleep30-21swapper/308:22:113
1642727321,16sleep30-21swapper/308:22:113
1606927320,26sleep10-21swapper/110:02:281
1278027319,21sleep00-21swapper/009:52:250
967027220,33sleep09679-21modprobe08:01:560
70727221,33sleep0740-21grep07:32:190
30772727,11sleep00-21swapper/011:06:170
2610927214,22sleep20-21swapper/212:17:282
1525027217,24sleep20-21swapper/208:17:212
1454527212,42sleep114692-21unixbench_multi09:57:271
1113427212,47sleep2281-21dbus-daemon11:31:562
2114627117,27sleep00-21swapper/008:36:580
1981327111,45sleep219880-21/usr/sbin/munin08:32:112
1380627115,26sleep10-21swapper/108:12:271
683627022,27sleep10-21swapper/111:17:491
510727015,40sleep15248-21modprobe07:46:581
2727427011,43sleep127308-21latency12:22:201
1122027016,41sleep111302-21mailstats09:47:261
103122707,11sleep00-21swapper/009:45:190
504926918,21sleep00-21swapper/011:12:150
504926918,21sleep00-21swapper/011:12:150
427326916,17sleep10-21swapper/109:26:581
2700626912,16sleep30-21swapper/312:22:133
226442697,12sleep10-21swapper/108:41:561
62022686,12sleep20-21swapper/207:47:322
3205826822,29sleep132129-21/usr/sbin/munin07:27:271
3203626819,33sleep232067-21latency_hist09:11:572
1425526814,42sleep014257-21cron11:41:550
99626718,24sleep11135-21tail10:57:291
310892677,13sleep00-21swapper/012:33:290
310892677,13sleep00-21swapper/012:33:290
2349326720,29sleep323546-21/usr/sbin/munin07:02:183
2349326720,29sleep323546-21/usr/sbin/munin07:02:183
1823026716,32sleep218247-21awk08:27:062
1308626722,26sleep113028-21latency_hist08:11:591
1150226711,24sleep00-21swapper/009:47:290
771826619,29sleep17806-21tune2fs09:37:161
2893826616,24sleep10-21swapper/109:01:581
203926614,21sleep10-21swapper/109:17:251
1984426620,26sleep10-21swapper/110:16:581
736526515,27sleep06808-21/usr/sbin/munin07:52:270
2898926517,31sleep229073-21sed07:17:272
2303726517,24sleep00-21swapper/010:26:590
1700526515,30sleep20-21swapper/208:22:262
1700526515,30sleep20-21swapper/208:22:262
1150926514,35sleep111572-21cut08:06:581
88012649,26sleep10-21swapper/111:22:261
873626417,28sleep20-21swapper/211:22:222
3046826413,36sleep230496-21munin-run09:06:562
3046826413,36sleep230496-21munin-run09:06:562
1980726411,36sleep30-21swapper/308:32:083
459526319,25sleep10-21swapper/109:27:141
3226826310,41sleep00-21swapper/010:54:330
3226826310,41sleep00-21swapper/010:54:330
2746426317,22sleep00-21swapper/007:12:280
2133826316,22sleep20-21swapper/210:21:562
194282637,15sleep20-21swapper/211:57:162
655426217,27sleep16247-21apt-get11:17:001
655426217,27sleep16247-21apt-get11:17:001
46626210,25sleep00-21swapper/009:12:270
3004826211,28sleep00-21swapper/007:22:130
21932626,12sleep00-21swapper/007:37:120
21932626,12sleep00-21swapper/007:37:120
1978426212,26sleep10-21swapper/108:32:061
1242426224,18sleep20-21swapper/208:07:272
710326114,21sleep10-21swapper/107:52:191
528826116,27sleep05284-21apt-get07:47:090
2935626116,26sleep229373-21/usr/sbin/munin10:46:562
243382615,45sleep00-21swapper/007:04:000
243382615,45sleep00-21swapper/007:04:000
2001426117,26sleep120113-21users11:57:271
1656926116,26sleep00-21swapper/011:47:250
1656926116,26sleep00-21swapper/011:47:250
161172615,10sleep10-21swapper/108:21:401
1188626120,20sleep10-21swapper/111:32:231
1142326115,25sleep00-21swapper/011:32:100
292612608,13sleep00-21swapper/009:02:140
292612608,13sleep00-21swapper/009:02:140
2146426013,19sleep00-21swapper/008:37:140
1017326018,17sleep00-21swapper/008:02:080
3073725915,19sleep10-21swapper/109:07:111
2652325915,26sleep226542-21apt-get10:37:062
2162725917,26sleep121620-21apt-get10:22:111
148902596,15sleep20-21swapper/209:59:022
59452586,11sleep20-21swapper/211:13:222
59452586,11sleep20-21swapper/211:13:222
410825813,29sleep04136-21/usr/sbin/munin07:42:210
251225813,22sleep20-21swapper/211:02:272
245125811,19sleep20-21swapper/207:37:182
245125811,19sleep20-21swapper/207:37:182
235282585,44sleep00-21swapper/012:11:200
1874625824,16sleep10-21swapper/108:27:381
830825715,18sleep00-21swapper/011:22:080
683525713,30sleep26866-21grep07:52:122
373825716,24sleep13735-21apt-get07:42:101
243925715,22sleep10-21swapper/111:02:261
2376425715,23sleep123770-21ntp_states07:02:241
2376425715,23sleep123770-21ntp_states07:02:241
2008125717,23sleep120075-21apt-get10:17:101
1543525715,22sleep00-21swapper/008:17:270
667225612,27sleep16795-21timerandwakeup09:32:271
2974125613,29sleep129812-21cat07:21:581
2496825614,25sleep024964-21apt-get10:32:080
2496825614,25sleep024964-21apt-get10:32:080
2275225614,17sleep10-21swapper/108:42:001
166382568,12sleep10-21swapper/108:22:171
166382568,12sleep10-21swapper/108:22:171
138525615,19sleep20-21swapper/209:17:062
1304825612,30sleep013084-21cat08:11:580
1227325615,24sleep112268-21apt-get09:52:101
341525512,25sleep03409-21apt-get11:07:100
2603125518,22sleep126108-21sort12:17:271
1496125514,17sleep10-21swapper/111:42:211
1033125513,26sleep010337-21ntp_states11:27:210
187225415,23sleep11901-21munin-run07:36:571
1636325413,17sleep20-21swapper/211:47:182
1636325413,17sleep20-21swapper/211:47:182
148425414,20sleep00-21swapper/009:17:120
762725315,21sleep27624-21apt-get09:37:092
744025313,17sleep00-21swapper/009:36:590
496325311,17sleep00-21swapper/009:27:220
2909425312,20sleep20-21swapper/212:27:252
2771825314,23sleep227751-21users07:12:302
2700125313,24sleep026996-21apt-get12:22:100
987725213,22sleep19944-21df11:27:101
731325212,24sleep27387-21systemctl09:36:572
337825214,22sleep23415-21ntp_states09:22:232
308872527,11sleep10-21swapper/110:51:441
3050225211,22sleep230549-21users10:47:292
2555225214,18sleep20-21swapper/210:32:262
2555225214,18sleep20-21swapper/210:32:262
2362325213,16sleep10-21swapper/112:11:571
1850625212,15sleep00-21swapper/011:52:300
1020025213,23sleep210193-21apt-get08:02:102
69125114,20sleep20-21swapper/207:32:172
2615825113,24sleep126203-21unin-run07:11:561
2547225115,19sleep10-21swapper/110:32:251
2547225115,19sleep10-21swapper/110:32:251
1337325112,16sleep00-21swapper/008:12:140
129702517,32sleep20-21swapper/211:37:052
3118125011,22sleep031192-21ntp_states09:07:230
2162825012,21sleep221696-21df10:22:122
116025017,15sleep10-21swapper/109:16:571
1098625013,22sleep210999-21irqstats09:47:192
3227924912,19sleep20-21swapper/209:12:092
2098924910,20sleep221038-21irqstats12:02:172
1336824916,16sleep10-21swapper/111:37:201
431524810,15sleep20-21swapper/207:42:272
3123324812,21sleep231268-21df10:52:132
3123324812,21sleep231268-21df10:52:132
2828324816,16sleep20-21swapper/208:57:272
218924813,19sleep12186-21apt-get07:37:091
218924813,19sleep12186-21apt-get07:37:091
3024624710,17sleep10-21swapper/107:22:181
2941124710,19sleep129452-21users12:27:291
2525324711,16sleep10-21swapper/112:16:581
2768724610,21sleep027699-21df_inode08:57:130
2386224611,17sleep20-21swapper/212:12:112
2654824410,17sleep10-21swapper/110:37:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional