You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-11-17 - 14:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Tue Nov 17, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
174082156426,1521sleep017410-21ntp_states22:34:390
875227097,689sleep38766-21grep20:24:333
31298246816,437sleep131244-21latency_hist19:54:141
26572468113,338sleep12656-21ntp_states20:04:401
16533242759,217sleep30-21swapper/320:49:333
4709236411,338sleep04715-21ntpq21:54:430
2129423628,338sleep321405-21sed19:20:043
2129423628,338sleep321405-21sed19:20:043
184672362107,190sleep318589-21latency_hist19:14:153
1929923569,335sleep319340-21ntpq19:15:203
1373422798,222sleep313768-21ntpq20:39:393
1373422798,222sleep313768-21ntpq20:39:393
10376227630,155sleep310433-21ls20:29:343
1606226850,199sleep31692-21apt-key23:29:133
647226619,174sleep30-21swapper/319:59:293
28902223710,210sleep328954-21cpuspeed21:29:303
18910223717,207sleep018919-21vmstat20:54:470
14938223350,146sleep315046-21fschecks_time19:04:333
14938223350,146sleep315046-21fschecks_time19:04:333
115952230190,24sleep111658-21sshd23:59:321
115952230190,24sleep111658-21sshd23:59:321
2955522249,203sleep3201-21jbd2/mmcblk2p2-19:47:383
25516222139,165sleep325541-21munin-run21:19:123
14838220821,49sleep00-21swapper/019:04:260
14838220821,49sleep00-21swapper/019:04:260
8332220547,118sleep38479-21unixbench_singl22:04:433
13145220310,175sleep313205-21idleruntime00:04:313
12756220138,60sleep30-21swapper/322:19:413
774421976,157sleep00-21swapper/022:04:320
23095218740,60sleep31-21systemd22:54:123
23095218740,60sleep31-21systemd22:54:123
18082218749,24sleep30-21swapper/320:54:293
8384217410,100sleep30-21swapper/323:49:323
2754217222,129sleep12794-21sort21:49:311
31074216711,140sleep231152-21find21:34:422
28738216710,129sleep30-21swapper/323:09:433
2972621656,13sleep20-21swapper/223:14:302
20537216417,126sleep320663-21latency_hist00:29:143
5204216350,54sleep30-21swapper/323:39:273
6963216111,134sleep37025-21grep20:19:323
6511216117,130sleep16492-21latency_hist23:44:131
10054216117,107sleep310115-21idleruntime23:54:333
2754321577,100sleep30-21swapper/319:39:433
22934215330,65sleep3367-21snmpd21:09:353
16502215310,43sleep30-21swapper/300:14:363
8060214917,117sleep18030-21latency_hist23:49:141
8060214917,117sleep18030-21latency_hist23:49:141
1198121499,123sleep311984-21cron22:19:123
22570214610,123sleep022593-21netstat19:24:380
22570214610,123sleep022593-21netstat19:24:380
2212214414,109sleep30-21swapper/323:29:383
2212214414,109sleep30-21swapper/323:29:383
10117214421,107sleep010155-21irqstats23:54:340
1281421435,123sleep00-21swapper/000:04:150
960121409,94sleep39640-21wc22:09:403
2051214017,102sleep11567-21apt-get23:29:231
2051214017,102sleep11567-21apt-get23:29:231
6949213915,17sleep30-21swapper/321:59:593
6949213915,17sleep30-21swapper/321:59:593
10211213519,34sleep00-21swapper/022:11:420
20679213416,104sleep120680-21cpu19:19:461
20679213416,104sleep120680-21cpu19:19:461
18950213321,97sleep00-21swapper/019:14:320
2123213218,11sleep30-21swapper/321:45:543
668321318,98sleep30-21swapper/320:19:163
3047421308,101sleep30-21swapper/321:34:273
1645021300,14sleep212-21rcu_sched19:09:162
1645021300,14sleep212-21rcu_sched19:09:162
5207212818,10sleep30-21swapper/321:55:183
14595212717,96sleep01-21systemd20:44:120
14595212717,96sleep01-21systemd20:44:120
30662212416,11sleep30-21swapper/323:16:053
3283212315,10sleep30-21swapper/320:07:003
12048212227,80sleep312165-21timerandwakeup23:59:423
12048212227,80sleep312165-21timerandwakeup23:59:423
24232212136,42sleep10-21swapper/121:14:221
2280021209,53sleep30-21swapper/319:24:453
2280021209,53sleep30-21swapper/319:24:453
16715212044,27sleep30-21swapper/322:34:143
16715212044,27sleep30-21swapper/322:34:143
5809211922,24sleep20-21swapper/223:39:422
3793211627,78sleep3367-21snmpd21:52:283
23412211610,90sleep123464-21cpuspeed22:54:271
21252211315,65sleep20-21swapper/200:29:412
32161211116,78sleep132235-21fw_packets21:39:341
31616211116,78sleep031514-21/usr/sbin/munin19:54:300
31616211116,78sleep031514-21/usr/sbin/munin19:54:300
18938211017,19sleep20-21swapper/200:24:122
18938211017,19sleep20-21swapper/200:24:122
13650211020,74sleep213695-21ntpq18:59:362
21501210923,27sleep10-21swapper/100:29:441
2113221088,41sleep30-21swapper/321:04:313
12170210817,72sleep012318-21unixbench_singl23:59:430
12170210817,72sleep012318-21unixbench_singl23:59:430
4017210712,51sleep314624-21cyclictest20:09:393
25077210541,22sleep30-21swapper/319:34:143
672621049,51sleep30-21swapper/323:44:243
672621049,51sleep30-21swapper/323:44:243
32477210210,48sleep30-21swapper/321:39:403
26680210217,64sleep126610-21latency_hist19:39:161
14288210121,63sleep214348-21processes22:24:402
2550129913,70sleep225572-21sensors_temp22:59:412
954229810,76sleep0367-21snmpd23:53:580
2367229861,18sleep30-21swapper/322:54:333
312862978,54sleep331378-21perl23:19:323
2651529730,13sleep30-21swapper/323:04:263
2113129632,39sleep221123-21apt-get21:04:302
206622965,80sleep30-21swapper/321:02:043
1497429618,57sleep115011-21fschecks_count20:44:301
1497429618,57sleep115011-21fschecks_count20:44:301
197112955,79sleep30-21swapper/322:40:323
197112955,79sleep30-21swapper/322:40:323
997329317,62sleep010033-21latency_hist20:29:140
925129315,63sleep19338-21cat20:24:511
3138029315,60sleep00-21swapper/023:19:330
2706229332,40sleep30-21swapper/321:24:133
2420429313,60sleep124210-21/usr/sbin/munin19:29:391
2204329220,50sleep0231-21systemd-journal00:34:110
3065129120,54sleep230809-21unixbench_multi19:49:472
2384329016,56sleep223934-21fschecks_time19:29:322
435828921,43sleep24513-21grep23:34:502
1744128924,44sleep017526-21cat00:19:130
1298628816,59sleep212979-21apt-get00:04:252
2967628723,40sleep029697-21awk19:49:130
825728634,24sleep10-21swapper/120:24:151
1288728627,44sleep113009-21timerandwakeup22:19:431
1155028618,52sleep011691-21latency_hist20:34:150
448928519,49sleep14640-21head20:09:491
972228416,46sleep19114-21/usr/sbin/munin22:09:421
551228427,35sleep10-21swapper/121:59:121
2533028420,33sleep00-21swapper/019:34:310
2140928422,43sleep021543-21unixbench_singl19:20:050
2140928422,43sleep021543-21unixbench_singl19:20:050
1631428421,33sleep10-21swapper/119:09:161
1631428421,33sleep10-21swapper/119:09:161
1072928413,52sleep110793-21cpuspeed22:14:301
1583728316,45sleep115349-21/usr/sbin/munin22:29:401
1583728316,45sleep115349-21/usr/sbin/munin22:29:401
2496528220,46sleep30-21swapper/322:59:253
1870528215,52sleep018225-21apt-get22:39:190
1870528215,52sleep018225-21apt-get22:39:190
1673128220,40sleep116812-21awk00:14:421
2712028116,50sleep227200-21timerandwakeup23:04:412
2532828119,47sleep30-21swapper/319:34:303
189162814,68sleep00-21swapper/000:23:540
189162814,68sleep00-21swapper/000:23:540
470628021,43sleep30-21swapper/323:36:323
1381828020,45sleep30-21swapper/322:24:253
842327911,52sleep28608-21apt-key20:24:162
78327912,53sleep2865-21mailstats23:24:402
342327917,48sleep03473-21latency_hist23:34:150
342327917,48sleep03473-21latency_hist23:34:150
2848727919,11sleep30-21swapper/321:27:363
2077327932,32sleep30-21swapper/300:29:263
186062795,63sleep30-21swapper/300:19:573
186062795,63sleep30-21swapper/300:19:573
1525827911,34sleep20-21swapper/220:44:352
1525827911,34sleep20-21swapper/220:44:352
557427813,50sleep05605-21/usr/sbin/munin23:39:360
1592127819,10sleep30-21swapper/320:46:173
1592127819,10sleep30-21swapper/320:46:173
1309627810,17sleep30-21swapper/320:39:153
663627714,17sleep20-21swapper/220:19:152
2249127717,40sleep122618-21grep22:49:461
2249127717,40sleep122618-21grep22:49:461
2037327732,30sleep30-21swapper/322:44:313
2037327732,30sleep30-21swapper/322:44:313
77182768,14sleep00-21swapper/020:19:450
3276127618,11sleep30-21swapper/319:58:363
3276127618,11sleep30-21swapper/319:58:363
3095327619,10sleep30-21swapper/319:50:453
263232760,8sleep10-21swapper/123:04:151
174827619,36sleep01831-21find21:45:060
1435027627,32sleep30-21swapper/300:09:153
1435027627,32sleep30-21swapper/300:09:153
1796527529,11sleep30-21swapper/322:36:033
2964727413,35sleep10-21swapper/123:14:251
208327425,26sleep20-21swapper/223:29:282
208327425,26sleep20-21swapper/223:29:282
1965827421,36sleep219796-21sed19:15:272
1643727421,32sleep216431-21apt-get20:49:272
62052735,49sleep20-21swapper/221:59:462
62052735,49sleep20-21swapper/221:59:462
601127316,28sleep00-21swapper/021:59:230
601127316,28sleep00-21swapper/021:59:230
530327318,24sleep00-21swapper/020:14:240
400827323,31sleep00-21swapper/021:54:140
2806327310,17sleep00-21swapper/023:09:280
2743727318,35sleep127513-21cut21:24:311
813627218,23sleep10-21swapper/122:04:411
2354927217,39sleep123698-21sed19:29:151
193312720,8sleep10-21swapper/122:40:001
193312720,8sleep10-21swapper/122:40:001
1202727215,41sleep212062-21irqstats20:34:372
2505227117,27sleep00-21swapper/022:59:300
2349527111,41sleep023513-21fschecks_count22:54:300
1460727111,44sleep014616-21df_inode00:09:280
1460727111,44sleep014616-21df_inode00:09:280
2806727027,27sleep20-21swapper/221:24:452
2762227024,26sleep00-21swapper/021:24:340
2864426918,33sleep228737-21/usr/sbin/munin23:09:422
243226924,29sleep10-21swapper/121:49:151
149192696,11sleep00-21swapper/022:27:040
1333626919,25sleep10-21swapper/120:39:291
1333626919,25sleep10-21swapper/120:39:291
3250226818,26sleep20-21swapper/221:39:432
251712687,12sleep00-21swapper/021:14:450
2069126825,24sleep10-21swapper/122:44:371
2069126825,24sleep10-21swapper/122:44:371
513026721,21sleep00-21swapper/020:14:160
2270826714,32sleep222798-21mailstats19:24:442
2270826714,32sleep222798-21mailstats19:24:442
1825526718,28sleep218277-21/usr/sbin/munin22:39:132
1674926718,25sleep00-21swapper/019:09:190
1514826725,24sleep10-21swapper/100:09:411
1514826725,24sleep10-21swapper/100:09:411
75826618,32sleep1782-21/usr/sbin/munin23:24:391
2075326618,31sleep020794-21proc_pri22:44:400
2075326618,31sleep020794-21proc_pri22:44:400
1934726613,37sleep119434-21/usr/sbin/munin00:24:311
1621726616,28sleep216253-21/usr/sbin/munin00:14:312
3200226513,36sleep232042-21ps19:54:402
3200226513,36sleep232042-21ps19:54:402
2589126524,20sleep20-21swapper/221:19:322
1764326520,24sleep10-21swapper/119:09:531
1649426512,22sleep00-21swapper/020:49:290
1081226517,32sleep010847-21fschecks_count22:14:330
603226413,34sleep16030-21apt-get21:59:241
603226413,34sleep16030-21apt-get21:59:241
2196526418,21sleep20-21swapper/222:49:352
2196526418,21sleep20-21swapper/222:49:352
18182648,12sleep20-21swapper/220:03:042
1517026416,21sleep10-21swapper/122:29:151
1515726418,27sleep00-21swapper/020:44:330
1515726418,27sleep00-21swapper/020:44:330
1370426419,20sleep20-21swapper/220:39:382
1370426419,20sleep20-21swapper/220:39:382
83226316,23sleep00-21swapper/019:59:340
3084426312,28sleep00-21swapper/021:34:360
3022726320,21sleep20-21swapper/221:34:142
2572926314,32sleep225770-21ntpq19:34:412
2307326318,17sleep20-21swapper/221:09:382
2089326311,36sleep021008-21sed21:04:150
1363226314,32sleep213778-21cut22:24:162
1082026314,31sleep110187-21/usr/sbin/munin20:29:441
551826220,23sleep25539-21munin-run21:59:132
2055726212,32sleep220573-21/usr/sbin/munin22:44:342
2055726212,32sleep220573-21/usr/sbin/munin22:44:342
1922726219,25sleep00-21swapper/000:24:260
1705026218,26sleep217030-21apt-get19:09:372
1524426223,20sleep20-21swapper/200:09:422
1524426223,20sleep20-21swapper/200:09:422
111242628,12sleep20-21swapper/222:14:392
3015626118,24sleep10-21swapper/121:34:121
2786026118,20sleep00-21swapper/023:09:150
1534826117,27sleep015367-21apt-get22:29:220
1534826117,27sleep015367-21apt-get22:29:220
1024026117,20sleep20-21swapper/223:54:362
891426016,27sleep29030-21sed23:49:442
702426014,19sleep20-21swapper/223:44:352
702426014,19sleep20-21swapper/223:44:352
266326019,22sleep20-21swapper/220:04:412
230112609,29sleep10-21swapper/119:24:471
2160826020,21sleep00-21swapper/021:04:440
1940826020,18sleep20-21swapper/222:40:022
1940826020,18sleep20-21swapper/222:40:022
1225426012,25sleep00-21swapper/020:34:440
1029926017,17sleep20-21swapper/220:29:332
67182596,40sleep00-21swapper/023:44:160
67182596,40sleep00-21swapper/023:44:160
532525918,24sleep15322-21apt-get20:14:251
427325918,26sleep14357-21expr23:34:491
3146625915,28sleep20-21swapper/223:19:352
2304425916,26sleep123072-21/usr/sbin/munin21:09:371
1695625921,18sleep10-21swapper/122:34:281
1250525913,26sleep00-21swapper/022:19:340
40425810,32sleep10-21swapper/119:59:151
40425810,32sleep10-21swapper/119:59:151
2845925817,21sleep10-21swapper/123:09:371
2154025814,27sleep121585-21ntp_states21:04:411
1577125815,25sleep20-21swapper/222:29:372
1577125815,25sleep20-21swapper/222:29:372
1369725822,19sleep30-21swapper/318:59:373
767425718,23sleep27675-21diskstats22:04:302
417425716,25sleep24198-21ntp_states20:09:422
2744525717,18sleep20-21swapper/219:39:422
2587625716,25sleep00-21swapper/021:19:300
219525716,25sleep02189-21apt-get20:04:270
1919125712,21sleep10-21swapper/119:15:181
1371125712,18sleep00-21swapper/018:59:380
913925612,26sleep29133-21apt-get22:09:262
695625618,22sleep26960-21diskstats20:19:312
2889825613,25sleep028894-21apt-get21:29:270
2705025613,18sleep10-21swapper/123:04:401
1304525615,26sleep113073-21munin-run20:39:131
988525513,23sleep19879-21apt-get23:54:271
3250725516,22sleep10-21swapper/123:24:151
2095025518,22sleep20-21swapper/219:19:552
2095025518,22sleep20-21swapper/219:19:552
1800525514,24sleep218007-21apt-get20:54:242
1274125519,20sleep21-21systemd00:04:132
1274125519,20sleep21-21systemd00:04:132
83625420,18sleep00-21swapper/021:44:130
2972625417,23sleep20-21swapper/219:49:152
2961525412,30sleep00-21swapper/023:14:160
2707625419,17sleep10-21swapper/119:39:331
2415825416,18sleep00-21swapper/019:29:380
2340825414,24sleep223405-21apt-get22:54:242
2227725411,18sleep00-21swapper/022:49:410
2227725411,18sleep00-21swapper/022:49:410
1993625421,16sleep10-21swapper/120:59:361
1929625412,17sleep20-21swapper/200:24:282
72025314,24sleep1731-21diskstats19:59:311
286282539,24sleep10-21swapper/119:44:331
14682535,37sleep00-21swapper/023:28:050
829725213,22sleep18292-21apt-get23:49:261
314772525,35sleep10-21swapper/121:35:541
2996625215,19sleep00-21swapper/019:49:290
1458625214,17sleep10-21swapper/119:04:151
136125213,17sleep20-21swapper/221:44:562
1856625111,17sleep10-21swapper/120:54:431
269762508,15sleep00-21swapper/019:39:320
266025014,21sleep22663-21apt-get21:49:242
1996425013,21sleep220002-21ntpq20:59:372
1325624915,19sleep10-21swapper/100:04:331
254422486,34sleep20-21swapper/221:18:142
1701124818,14sleep10-21swapper/120:49:431
262582477,14sleep10-21swapper/121:19:391
2276024714,20sleep022772-21forks21:09:320
491524311,20sleep04945-21munin-run23:39:120
177552408,15sleep10-21swapper/100:19:311
177552408,15sleep10-21swapper/100:19:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional